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1. INTRODUCTION 


Optimal control theory of stochastic linear systems has advanced 
rapidly since the original contributions [1], [2]. For discrete-time 
linear dynamic systems with additive Gaussian noise, the true optimality 
is guaranteed by the Separation Theorem [3], [4], However, nearly all 
of the results have been obtained under the assumption of the existence 
of a centralized decision maker which takes all the measurements and 
generates all control inputs. For systems with multiple decision makers 
a general computationally feasible theory does not exist at the present 
time. On the other hand, since most system designs and Implementations 
have been simple and analog in nature, there is a lag in the utilization 
of digital components and the correspondingly improved performance that 
could be achieved. This can be partially explained in that in many 
cases the designers of a control system usually have constraints which 
make a decentralized computer approach luqir act leal. 

The availability of microprocessors, assembled from a small set 
of LSI logic components, has presented the control system designer with 
new opportunities for sophisticated control system design. Many de- 
signers of control system components have improved their products 
through the use of microprocessors. This fact is substantiated in re- 
ference [5] where a design of a microprocessor-controlled rate gyro is 
presented. The use of a microprocessor in the rate gyro resulted in 
an improved performance and a substantial savings in hardware. Incor- 
poration of microcomputers also makes redtindant navigation systems avail- 
able for only a moderate cost and size increase over non-redundant sys- 
tems. The trend of device manufacturers to use microcomputers as 
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integral parts of their equipment is expected to continue. The features 
of a microprocessor — flexibility, modularity, good hardware communica- 
tions and low cost, have made distributed or decentralized control a 
suitable answer to many control problems. A distributed-control system 
offers several potential advantages over a centralized control system, 
but in the past the high cost of processors has discouraged its use. 

The advances in microprocessor technology have shifted and are continu- 
ing to shift the cost balance in favor of distributed control systems. 
The following are some of the potential advantages of a distributed- 
control system. 

1. The interface between the computer and the process, i.e., 
the wiring of the process input and output signals to a centralized' 
computer has constituted a considerable portion of the toL'^l cost for 
a centralized control system. In some cases the wiring is susceptible 
to electromagnetic interferences. Communication difficulties may arise 
because of non-instantaneous transmission of data between physical lo- 
cations; by the time the control computer receives the data it may be 
too late for effective control. Therefore, it is desirable to bring 
the computers as close as possible to the process. 

2. The computation time required to process the data in real 
time at a centralized computer may be great despite the rapid advance 

in mainframe computer technology. Intolerable time delays may occur be- 
tween a portion of the process needing attention and another part which 
has information for the required action. Parallel computation, with 
microcomputers each dedicated to a portion of the process, can help to 
solve this type of problem. 
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3. Computer down-time has been a nightmare in real-time appli- 
cations. In most cases it has been so intolerable that a back-up unit 
is required to ensure reliable operation. But idle facilities can be 
very hard to justify economically. The structure of a distributed con- 
trol system is more reliable in the sense that failure in one computer 
does not usually have catastrophic consequences . For the portion of the 
process which is critical to the whole operation, a redimdant microcom- 
puter can be Installed for reliable control at a minimal cost Increase. 

4. The initial capital investment for centralized control may 
not be economically feasible because of the high cost of mainframe com- 
puters. A distributed control system installed step by step, can be 
more attractive to the management when there is a constraint on the 
capital expenditure. 

Information exchange is a critical issue in distributed-control 
systems. In a distributed-control system, it must be assumed that no 
controller, local or central, possesses a complete description of the sys' 
tern. Figure (1.1) shows an example of a distributed flight control sys- 
tem with minimal module requirements from the flight control point of 
view. Note that a microcomputer controller is associated with each physi- 
cal device in the system and that each controller can communicate with 
other physically distributed elements using an information exchange bus. 
The controllers, therefore, must exchange information among themselves 
in order to achieve satisfactory opera n. The type of information 
available to a controller for its decision-making is called its infor- 
mation pattern [6], [7]. Hence, for a distributed processing system we 
have an information pattern considerably different from that of a 
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CU = Control Unit 


Figure 1.1 A Distributed Flight Control System 







5 


centralized control system in which the central controller is assumed 
to have oomptete and instantaneous information of the system. The 
following are some practical aspects we must consider in a distributed 
microprocessor-based control system. 

1. Because of the high cost of communication, various techniques 

for data reduction are needed. For example, in a distributed aircraft 
control system, an adaptive sampling technique may be employed to re- 
duce the data traffic between sensors and actuators [8] , [9] , [10] . In 
an adaptive sampling system, the trade-offs between performance and 
communication loads must be considered. The current estimate of the 
state is transmitted to the actuator only as needed to maintain ade- 
quate performance of the system. Thus, there will be variable time in- 
tervals between information updates to the actuators. , 

2. As more transmission lines are added to a distributed-control 
system, it becomes economical to multiplex data on the lines. In some 
cases, a reduction In weight and size Is also a significant improvement. 
This is especially true in aerospace applications. There are two 
common methods to merge data over a single transmission line — Frequency 
Division Ifiiltiplexing (FDM) and Time Division Multiplexing (TDM) . In 
FDM, devices share a common transmission line by dividing its frequency 
spectrum into several subchannels. In TDM, devices queue on one end of 
the transmission line so that exactly one device is allowed to trans- 
mit on it at a time. In the latter case , the data arrival time may be 
uncertain. 

3. For economic as well as system stability reasons, the sampling 
rate should be selected according to the bandwidth of the loop being 
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controlled. If loops can be controlled separately, they can usually be 

sampled at different rates. The necessity for a multivariate system be- 
comes obvious if the bandwidths are more than an order of magnitude apart. 

The advantages of multirate systems are so great that they have been 
employed in industry even though there are no complete analytic tech- 
niques to design them at the present time [11], [12]. One example is in 
the Digikon-II program of the space Shuttle flight control system, where 
multirate results are developed for 2:1 and 4:1 sampling ratios [13]. 

The above discussion indicates that a distributed control system 
may have a stochastic information pattern where the data arrives at 
random times. It is surprising that modern control theory has only a 
limited amount of available work on this practical aspect of control 
problems, whereas single rate control and filter problems are well de- 
veloped [2], [4]. However, it is encouraging that recently, there has 
been an increasing interest in multirate systems. Primarily, though, it 
has been assumed that the information pattern is deterministic in both 
multirate and single rate cases; and control systems with stochastxc 
information patterns have not received enough attention. 

In this report, a stochastic model of a sampling process is postu- 
lated which accounts for 

(1) Variable time intervals in adaptive-sampling systems 

(2) Uncertainty of data arrival time in multiplexed communi- 
cation systems 

(3) Multirate-sampling systems 

Stochastic modeling of a sampling process was first suggested by 
A. K. Caglayan and H. F. VanLandingham [14]. In reference [14] an 
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optimal control policy was obtained for the system with a discrete 
Markovian sampling process. However, the optimal solution was restricted 
to a process In idilch the time Interval between each Information update 
was known to the system at the Initiation of the interval. In this 
report an optimal policy is obtained for virtually all Markov type 
sampling processes. The restriction was relaxed to include the situa- 
tion where the control unit does not have to know the duration of the 
information update interval at its initiation, but instead, only has 
knowledge of the past history of the information update intervals. This 
assumption is more applicable to a distributed-control system because 
its control units ordinarily will not know how long they will have to 
act with the same information. The system with additive plant and 
measurement noise is, of course, also considered as being a practical 
constraint of control systems. A separation theory of estimation and 
control for the randomly- samp led system is presented. In particular, in 
a randomly-sampled system the stochastic optimal control can be synthe- 
sized by cascading an optimal estimator with the optimal control law. 

This is significant because it considerably simplifies the implementa- 
tion of the stochastic optimal control of randomly sampled systems. The 
theory is applied to the control of the longitudinal motion of the 
F8-DFBW (F-8 Digital Fly-by-Wire) aircraft. Both theoretical and simu- 
lation results indicate that, for the application example, the optimal 
cost obtained using a variable time -increment Markov information update 
process (where the controllers know only the past information update 
intervals and the Markov transition mechanism) is almost identical to 
the cost obtained using a known uniform update interval. 


2. OPTIMAL CONTROL OF RANDOMLY-SAMPLED SYSTEMS 


2. 1 Introduction 

The low cost of microprocessors together with increasingly complex 
control systems have necessitated a re-examination of the information 
structure in conventional Linear-Quadratic-Gauss Ian (LQG) controller de- 
sign. The conventional LQG control methods avoid consideration of any 
variation in information structure; it always assumes that the system is 
uniformly sampled, i.e., the information is transferred at fixed times 
with uniform Intervals. In a computer-based control system, especially 
those with multiple processors, it is not always possible or practical 
to transfer Information uniformly. A more general formulation is needed. 

A linear sampled-data control system can be classified by its input charac- 
teristics as having either random or deterministic inputs; by the measure- 
ment model as being noisy, when it is impossible to determine the state 
of the system exactly, or noise-free, when the state of the system can 
be known exactly; by the information transfer mechanism as being randomly- 
sampled, when the information is transferred at random times, or uniformly 
sampled when the information is transferred at fixed times with uniform 
intervals. In this chapter it is assumed that the input is deterministic 
and the measurement process is noise-free, but that the information is 
transferred at random times. A more general model with additive plant 
and measurement noise will be discussed in Chapter 4. 

A mathematical model describing the information transfer mechanism 
is first postulated in Section 2.2. Using dynamic programming, a control 
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law is then derived in Section 2.3 which optimizes the average of an 
integral of a quadratic form in the state and control variables. In 
section 2.4 we examine the asymptotic behavior of the optimal control 
for randomly -sampled systems. Conditions are given for the stability 
of the closed-loop systert using quasi-steady state feedback. 


2. 2 Stochastic Information Distribution Model 

The purpose of this section is to present a model which is useful 
in describing the stochastic information transfer process in a multiple- 
processor control system. The next section will present the derivation 
of the optimal control law that minimizes the average of an integral 
in the state and control variables. 

Consider the continuous dynamical system represented by the linear 
differential equation: 

X = Ax + Bu (2.2.1) 


where x is an n-dimensional vector, representing the system states, u 
is an m-dtmensional vector representing the control inputs, A is an 
n X n matrix describing the dynamics of the system and B is an n x m 
matrix describing the control effectiveness. It is assumed that the 
state Infoimatlon is transferred at discrete points in time, 
k “ 0,1,***,N where Oq ^k+1 *” 

The sequence is defined to be the sum of a stochastic pro- 
cess in the following sense: 


a 


k 


+ 


k 

Z t 
i»l 


i 


( 2 . 2 . 2 ) 



10 


where is an element of the set t(1). Hence, Oq denotes the 

starting time, t^ denotes the i~ time interval between two consecu- 
tive information updates and is the time that the information 

update takes place. Since the time interval between two consecutive in- 
formation updates cannot be negative, the sample space of the stochastic 
process t(i) for i ■ I,***, N, is restricted to sets of positive real 
numbers. There is, however, no restriction on the statistical nature 
of the process. The set of information update sequences which are ad- 
missible for a k-stage process is the Cartesian product of t(i), i. e. , 


T(k) e T(k) = t(1) x t(2) ••• x t(k) 


(2.2.3) 


tj^ e T (k) 

T(k) - {tj^, ^ 


(2.2.4) 


(2.2.5) 


Figure 2.2.1 illustrates the relation between t^ and a^. The 
average of a quadratic Integral in the state and control is defined to 
be the cost functional for the N-interval process to achieve the desired 
system performance. We assume a cost functional of the form: 


J - [x'^(t)Q x(t) + u'^(t)R u(t)]dt + x^(Oj,)Sx(0jj) 


( 2 . 2 . 6 ) 


The cost functional can be expressed as the sum of N Integrals by 
dividing the total time into N intervals, viz. 

. N— 1 _ ip >n 

J - J Z [x^(t)Q x(t) + u (t)R u(t)]dt + X (a )SxCa„) (2.2.7) 

2 T(N) . » " 
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It is interesting to note that the random variable appears in the 
limits of the integral only. Since the equivalent discrete-time system 
is obtained by integrating the system differential equation and cost 
functional over each sampling period, the random variable will appear 
everywhere in the equivalent discrete-time system! If control Inputs are 
further restricted to piecewise constant functions of time that change 
only at sa]^»ling instants i.e. 


u(t) - u(o^) 


"k i ‘ Vi ’ 


it is well known that 


( 2 . 2 . 8 ) 


xCt) - *(t, + I ♦(t.s) B(s)ds u(oj^) (2.2.9) 


Th^efora, the eqidvalent dlscrete^tlme system Is given by 

*^\+l^ “ I *^hc+l» (2»2.10) 


To simplify the notations, the following matrices are dctflned 

♦(k+1, k) - tj^) 

r^Tt^l 


r(k+l. k) 


♦ s)B(s)ds 


(2.2.U) 

( 2 . 2 . 12 ) 


Using equations (2.2J.0) and C2. 2*. 11), the. equations describing the pro- 
cess and cost function J can be written in difference equation form as 


x(k+l) - *(k+l, k) x(k) + r(k+l, k) v(k) 


N-1 


(2.2.13) 


" 7 Sx(N) + E* (x(k)'^ Q(k)x(k) + v(k)^R(k)v(k)}l(2.2.14) 

^ ' k-0 
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where 


At 

-1 T 

*(k+l, k) - e - R (k) W(k) 

(2.2.15) 

r(k+i ,k) - r(t^j^) 



ft 

r(t) ■ j e^Bds 

(2.2.16) 

Q(ic) - 

^ 0 

(e^)^Q e^dr - W(k)R“^J(k) 

(2.2.17) 


0 


H(k) - 

+ r^(r.0)Qr(r,0)]dr 

(2.2.18) 

4 

0 


W(k) ■ 

T Ar 

(e^)^Q (e^)dr 

(2.2.19) 


0 



The above representation was obtained using the techniques of 
Reference [15] with the transformation 

v(k) - u(k) + R(k)“^I(k)^x(k) (2.2.20) 

T 

employed to eliminate the cross product terms x (k)W(k)u(k) that would 


otherwise appear in the cost function representation. 
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2. 3 The Optimal Solution for Randomly-Sampled Systems 

We begin with a formal statement of the optimization problem for 
randomly~sampled systems: 

For the stochastic difference equation describing the system 

xOcfl) » *(W-1, k)x(k) + r(k+l, k)v(k) (2.3.1) 

where ♦(k+1, k) and TOcfl, k) are functions of tj^, tj^^^ and the cost 
function J given by 

N-1 

J - -I E{x(N)^Sx(N) + £ [x(k)'^Q(k)x(k) + v(k)\(k)v(k)]} (2.3.2) 

^ k-0 

we will select the sequence v(k) to minimize J given the distribution 
of the stochastic process 

The Principle of Optimality [16], [17] has proven to be a powerful 
tool for obtaining the solution of optimal control problems. This prin- 
ciple states that whatever the initial state and initial decision may be, 
the remaining decision must constitute an optimal policy with regard to 
the state resulting from the first decision. Using the principle of op- 
timality, we reduce the N-stage to N one-stage problems. This result is 
desirable because decisions can then be made sequentially. The optimi- 
zation is carried out using dynamic programming in a manner similar to 
that of Reference [18]. 

First we consider a single stage optimization problem over the last 
sample interval which starts at and terminates at a^. 

we will generalize the result to all N stages by induction. 


Subsequently 


15 


Single Stage Process 

Consider the last stage of the process which starts at and 

terminates at The optimal control over the last stage produces a 

cost 

J(N-1,N) = -kin E{x^(n-l)Q(N-l)x(N-l) + v(N-l)'^R(N-l)v(N-l) 
v(N-l) 

+ x(N)’^P(0)x(N)} (2.3.3) 

^ere P(0) ^ S. 

But x(N) is related to v(N-l) by the state eqxiation. 

x(N) » $(N, N-1) x(N-l) + r(N, N-1) v(N-l) 

Substituting this eqiiation into the cost function, we have 

J(N-1, N) - -y min E{x(N-l)k(N-l)x(N-l) + v^(N-l)R(N-l)v(N-l) + [$(N,N-1) 
v(N-l) 

x(N-l) + r(N, N-l)v(N-l)]'*’P(0)[$(N, N-l)x(N-l) + r(N, N-l)v(N-l) ] >(2.3.4) 

The control v(N-l) is required to be physically realizable and 
therefore should be Independent of the stochastic process t^. Using 
one of the properties of conditional expectation; namely, that E(x) = 
E[E(x/y)l where the outer expected value on the right-hand side is 
over y, we have 

J(N-1, N) -^min E{E^ * Q(N-l)x(N-l) + v’^(N-l)R(N-l)v CN-1) 

v(N-l) n' n-1 

+ [*(N, N-l)x(N-l) + r(N, N-1)v(N- 1)]’’^P(0)['KN, N-l)x(N-l) 

+ r(N, N-l)v(N-l)]>> (2.3.5) 


Vhe performance measure can be minimized by minimizing only the inner 


expected value in equation (2.3.5) with respect to v(N-l). The minimum 
is obtained by setting to zero. Thus, 

OV 

E {R(N-l)v(N-D + r(N, N-1)^P(0)($(N, N-l)x(N-l) 

"n Vi 

+ r(N, N-l)v(N-l)} - 0 (2.3.6) 

Since the stochastic process (t^) is independent of x(N-l), we have 
v(N-l) - -E , [R(N-D + r^(N, N-l)P(0)r(N, N-1)]"^ 

• E , [r^(N, N-1)P(0)4(N, N-l)]x(N-l) (2.3.7) 

n' N-1 

Introducing K(N-l) to represent the optimal gain at the (N-l)th stage, 
we have 

v(N-l) - K(N-l)x(N-l) (2.3.8) 

where 

K(N-l) - -E . [R(N-D + r’^(N, N-1) * P(0)r(N, N-1)]"^ 

V N-1 

E^ . [r’^(N, N-1)P(0)4(N, N-1)] (2.3.9) 

a N-1 

For notational simplification we let 

K(N-l) - K(N-1)/T(N-1) (2.3.10) 

i.e. , the conditioning will be understood in context. 

The cost resulting from application of the optimal control can be 
evaluated as 

J(N-1, N) -^E^ /_ {x^(N-l)Q(N-l)x(N-D + [K(N-l)x(N-l) 

^ ^n'Vi 

R(N-l)K(N-l)x(N-l) + [*(N, N-l)x(N-l) + (Continued) 
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+ r(N, N-l)K(N-l)x(N-l)]^P(0) [$(N, N-l)x(N-l) + r(N, N-1) 

K(N-l)x(N-l)] } (2.3.11) 

Hence, the cost function takes the foinn 

J(N-1, N) = ^ x(N~l)^ P(l) x(N-l) (2.3.12) 

where 

P(l) “ E {Q(N-D + K(N-1)^R(N-1)K(N-1) + [$(N, N-1) + P(N, N-1) 

■^n' N-1 

K(N-1)]'^P(0) * [»(N, N-1) + r(N, N-l)K(N-l)]} (2.3.13) 

For notatlonal simplification, let (as In 2.3.10) 

P(l) = P(l)/Vi (2.3.14) 

The result Is analogous to the optimal solution of uniformly-- 
sampled control systems. The optimal cost Is still a quadratic function 
of x(N-l), the Initial state for the single-stage problem; however, P(l) 
Is replaced by the mathematical expectation of the same function. This 
is an important characteristic of randomly -sampled control systems. 

We can no longer determine the minimum cost exactly, but Instead, 
only its statistical average before applying the control. 

Two Stage Process 

The final two Intervals of the process have an optimal cost 

1 T T 

J(N-2, N) - mln {x^(N)S x(N) + Z (x(k)''Q(k) 

v(N-l),v(N-2) N* N-l' N-2 K»N-2 


x(k) + v(k)^R(k)v(k))} 


(2.3.15) 
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= i min {E . • [x'^(N-2)Q(N-2)x(N-2) + v^(N-2)R(N-2) 

v(N-l),v(N-2) N* N-l' N-2 

v(N-2)] + E , [x^(N-l)Q(N-l)x(N-D 

N’ N-l' N-2 

+ v^(N-l)R(N-l)v(N-l) + x^(N)P(o)x(N)]} (2.3.16) 

J(N-2, N) » min E . {x^(N-2)Q(N-2)x(N-2) + v^(N-2)R(N-2)v(N-2) 

v(N-2) Vl'^N-2 

+ min E , [x’^CN-DQCN-DxCN-D + v^(N-l)R(N-l)v(N-l) 
v(N-l) ‘'n' N-1 

+ x^(N)P(0)x(N)]} (2.3.17) 

The last equation follows from the property of conditional distributions, 
viz. 





P P 

T /t T t /T 
n' N-l’ N-2 N-l' N-2 


s P P 

T /T t /T 
V N-l N-r N-2 


(2.3.18) 


From this property a similar identity can be derived for the mathemati- 
cal expectation operator. That is 


^ /T, (•) * E . [E , (.)] 

N’ N-l N-2 N N-2 n' N-l 


(2.3.19) 


From the Principle of Optimality, the cost functional for the two-stage 
problem can be written as 

J(N-2, N) - min E . {x^(N-’)Q(N-2) x(N-2) + v^(N-2)R(N-2)v(N-2) 

v(N-2) Vl' N-2 

+ J*(N-1, N)} 


(2.3.20) 









J(N-2, N) = -y min E . [x^(N-2)Q(N-2)x(N-2) + v^(N-2)R(N-2)v(n-2) 

^ Vl' N-2 

+ x’^CN-DPCDxCN-l)] (2.3.21) 

The cost functional for the two-stage process is analogous to that 
of the single stage process; v(N-2) can therefore be determined easily 
by repeating the steps which led to the expression for v(N-l). For the 
two-stage process* we conclude immediately that 

J(N-2, N) - x^(N-2)P(2)x(N-2) (2.3.22) 

P(2) = E {Q(N-2) + K^(N-2)R(N-2)K(N-2) + [$(N-1, N-2) + I’(N-1, 

'^N-r N-2 

N-2)K(N-2)]^P(1) • [$(N-1, N-2) + r(N-l, N-2)K(N-2)]} (2.3.23) 

where 

K(N-2) » -E [R(N-2) + r^(N-l, N-2)P(l)r (N-1, N-2)]"^ 

■^ N - l '\-2 

• E [r'^(N-l, N-2)P(1)<KN-1. N-2)] (2. 3. 24) 

^N-l' N-2 

where for notational simplification 

P(2) = ?(2)/Tjj_2 

and 

K(N-2) = K(N-2)/Tjj_2 (2.3.25) 

Utilizing the principle of induction, we can prove t'.<e theorem 
for the N-stage problem by carrying out the above derivation for h-stage, 
problem, assuming the form for the (k-1) stage problem, i.e. we will 
first assume that equations (2.3.22), (2.3.23) and (2.3.24) hold for the 
(k-l)-stage problem and then we will show it also holds for the k-stage 
problem. 
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Assumed Relations for the (k-l)-Sta8e Problem 

The optimal control at time N-k-t-1 for the (k-1) -stage process is 
characterized by the following equations. 

J(N-k+l, N) = ^ x'*^(N-lcfl)P(k-l)x(N-k+l) (2.3.26) 

P(k-l) - E , {Q(N-kfl) + K^(N-k+l)R(N-kfl)K(N-k+l) 

^N-k+2' Vk+1 

+ [$(N-k+2, N-k+1) + r(N-lc4-2, N-k+DKCN-kfl)]"*^ • P(l) 

[4(N-k+2, N-k+1) + r(N-k + 2, N-k+l)K(N-fc+l) ] } (2.3.27) 

and 

K(N-k+l) = -E , (R(N-k+l) + F^(N-k+2, N-k+1 )P(k-2)r(N-k+2, 

'^N-k+2' N-k+1 

N-k+1) • E . [r*^(N-k+2, N-k+l)P(k-2) 

‘^N-k+2' N-k+1 

• $(N-k+2, N-k+1)] (2.3.28) 

Using the Principle of optimality the cost function for the k-stage 
problem can be written as 

J(N-k, N) = min E , {- x"^(N-k) • Q(N-k)x(N-k) + ~ v(N-k)^ 

v(N-k) ’^N-k+l' N-k 

R(N-k)v(N-k) + J (N-k+1, N)} (2.3.29) 

Using equations (2.3.26j) and (2.3.1), we have 

J(N-k, N) - min E . {j x'^(N-k) • Q(N-k)x(N-k) + j v(N-k)’^ 

v(N-k) '^n-k+l' N-k 

R(N-k)v(N-k) + [<KN-k+l, N-k)x(N-k) 

+ r(N-k+l, N-k)v(N-k)]’*^ • P(k-l)['KN-k + 1, N-k)x(N-k) 

(Continued) 
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+ r(N-k+l, N)v(N-k) 


(2.3.30) 


The mlnliiiuin is obtained by setting 


aj(N-k. N) 
3v(N-k) ^ 


(2.3.31) 


Repeating the steps similarly used for the single-stage problem, 
the following recursive relations are obtained for the k-stage process 

J(N-k, N) - ^ x(N-k)^P(k) x(N-k) (2.3.32) 

P(k) - E , {Q(N-k) + K(N-k)^ • R(N-k)K(N-k) + [$(N-k+l, N-k) 

"^N-k+l'^N-k 

+ r(N-k + 1, N-k) • K(N-k)fp(k-l) [$(N-k+l, N-k) + T(N-k+l,N-k)K(N-k) ]} 

(2.3.33) 

and 

K(N-k) - -E [R(N-k) + r^(N-k+l, N-k)P(k-l)r(N-k+l, N-k)]"^ • 

“^N-k+l' N-k 

E . [r'^(N-k+l, N-k)P(k-l)*(N-k+l, N-k)] (2.3.34) 

"^N-k+l'^N-k 


Markov Process Assumption 

The statistical property characterizing a Markov process is analo- 
gous to the property lewtonian dynamics. In a dynamical system the 
equation of motion describing the future trajectory can be determined 
if the position and velocity are given at any time t; the past trajec- 
tory before time t is irrelevant. In a Markov process, the correspond- 
ing property is that the present state of the system contains all rele- 
vant statistics pertaining to the future. Mathematically, a process 
x(l), x(2),**’ is called a Markov process if [19], [20] 

P(x(N+l)/x(N) • ,x(D) - P(x(N+l)/x(N)) . 
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If * {t^ *** with Tj^ its sample space and if Tj^ is the 
sample space of t^, then assuming a Markov relationship. 


“ (2.3.35) 


This equation states that the present sampling interval determines the 

probability of the next sampling interval in the future. For such a 

process a transition mechanism can be defined and the joint density 

P„ (T, ) can be written as 
k ^ 


p? <v 

k 


\* ^k-1 ^ ^ 


k 

” n p (tk/Vi)P- (t^) 


k 

n p , (t./t, -,)P^ (t ) 


(2.3.36) 


Hence, for a Markov process , specification of the prior density 

function P_^ (t^) together with the transition probab? '' Ities 

P , it. /t. ,) completely determine the distribution of the process. 
^k'’^k-l ^ ^ 

As a result, equations ( 2 . 3 . 32 - 34 ) can be modified to replace condition- 
ing on Tjj ^ by conditioning on Hence, the probabilistic Riccati 

Equat?-:R& w.re 


J(N-k, N) - x^(N-k) P(k)x(N-k) 


(2.3J7 ) 
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P(k) - E , {Q(N-k) + K^(N-k) * R(N-k)K(N-k) + [$(N-k+l, N-k) 

^N-k+l'^N-k 

+ r(N-k+l, N-k)K(N-k)]^P(k-l) [4(N-k-l, N-k) + r(N-k+l, N-K)K(N-k) ] } 

(2.3.38) 

K(N-k) » -E . [R(N-k) + r(N-k+l, N-k)P(k-l)r(N-k+l, N-k)]^ • 

Vk+l'^N-k 

E , [r(N-kfl, N-k)P(k-l) • *(N-k4-l, N-k)] (2.3.39) 

'^N-k+l'^’N-k 

where 

P(k) - P(k)/Vj^ 

and 

K(N-k) = K(N-k)/Xj^_j^ 

The computation for the above probabilistic Rlccatl Equations Is 
simpler than the previous ones because the set that the expectation 
operators are condltloued on Is more restricted. However, If the ex- 
pectation operator is time-dependent and if the sample space of 
{tj^} is continuous, integration still has to be performed at each stage 
of the recursive Riccati equation. The computational time for this case 
can be exhorbitant. On the other hand, if the expectation operator is 
stationary (independent of time) and the sample space of {tj^} is con- 
tinuous, Integration for the expectation operator has to be performed 
only once before solving the probabilistic Riccati equations. A simpler 
case in which the expectation operator is stationary and the sample 
suace is discrete will be discussed in the next section. 
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Discrete Stationary Markov Processes 

If the sample space is discrete and finite and if the condition- 
al probability is stationary, then the transition mechanism can be re- 
presented by the matrix T ■ . * {T..} ■ (P/t, » S./t, , =» S.)} 

for a given sample space = {s^^, S 2 **‘* »Sjjg) for k » 1, ,N. In 

that case the optimal solution consists of NS gain and sensitivity 
matrices at each stage so that 


P(k)/Xjj_j^ e {P(k)/s^, P(k)/s 2 **• P(k)/Sjjg} (2.3.40) 

and 

K(N-k)/Xjj_j^ E {K(N-k)/sj^, K(N-k)/s2, K(N-k)/Sjjg} (2.3.41) 


The equation operator for a discrete sample space can be replaced 
by a summation. That is 

NS 

NS 

- 2 T., f(S^) (2.3.42) 

j-1 ^ 

The following notations will be used for simplification. 

Q(N-k, i) » » S^) = (2.3.43) 

R(N-k, i) - = S^) » (2.3.44) 

$(N-k, i) - “ ^i" “ *i (2.3.45) 

r(N-k, i) - r(tj,_k ’ ^1^ “ ^i (2.3.46) 

where Q^, R^, are defined in equation (3). The probabilistic 


eqviations can now be written in difference equation form. 
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J(N-k, N) » X (N-k)P(k/i)x(N-k) 


( 2 . 3 . 47 ) 


NS 


P(k/i) - Z T {Q(N-k, j) + K (N-k/j)R(N-k, j)K(N-k/J) + [4(N-k, j) 

j-1 ^ 

+ r(N-k, j)K(N-k/j)]'^P(k-l/j)l«(N-k, j) + F(N-k, j) * K(N-k/j)] 

( 2 . 3 . 48 ) 

NS , 

K(N-k/l) - - { Z T [R(N-k, j) + r^(N-k. 3) • P(k-l/j)r(N-k, j)] } 

j-1 


NS 

{ Z [r (N-k, m) • P(K-l/m)4(N-k, m)]} 

« im 


( 2 . 3 . 49 ) 


The recursive computations required to generate the optimal con- 
trol for the probabilistic Riccati equation above is similar to that 
for a deterministic Riccati equation. However, the probabilistic 
Riccati equation also requires the prior distribution function for 

t , viz. P (t ), to initiate the computation. Furthermore, at each 
o ■To ° 

stage of the recursive computation NS gain matrices and NS sensitivity 
matrices must be calculated. The computations required to generate the 
optimal control sequences for the randomly-sampled system is equivalent 
to solving NS Riccati equations coupled by transition probabilities. 

If the transition probability matrix is an identity matrix then the 
corresponding Riccati equations are decoupled. Each decoupled probabi- 
listic Riccati equation reduces to a deterministic Riccati equation with 
uniform sampling. The solution with N and k infinite is called the 
quasl-steady-state solution. Tu? quasi-steady solution is obtained as 


the NS gain matrices approach constant values. In general, the NS gain 
matrices have different quasi-steady values. For the purpose of com- 
puting quasi-steady-state gains, the prior distribution function can 


26 


be chosen to be 

^*^o “ V “ ^ (2.3.50) 

o 

and 

P, (t„ # S^) . 0 

O 

This is equivalent to selecting an initial sampling interval 
~ to start the process. Hence, there are generally NS simulations 
of interest in the class of distributions considered, one for each 
element of the discrete sample space x^. The optimal cost is not gener- 
ally asymptotically stationary. That is, the cost will depend on the 
initial sampling interval selected. One trivial example is the system 
with identity transition probability. The optimal cost depends on the 
initial sampling Interval chosen in this case because all subsequent 
sampling Intervals are the same as the initial sampling interval. Each 
simulation requires that the initial conditions x(o) and the initial 
sample interval t^ contained in x = {s^^ *** s^^g} be given. If the 
quasi-steady state gains are used and if t^ = S^, then the control 
u(o) applied over the interval tj^ is given by u(o) » K^x(o), where 
= K(N-k/i) for N and k infinite with i = 1, 2, •** NS. Subsequent 
control actions are computed according to u(k) = K^x(k) if t^^ = S^. 

Fig. (2.3.1) shows the closed loop control system using quasi-steady- 
state feedback. 

2. 4 The Stability of the Closed-Loop System 

In this section we will study the stability of the randomly-sampled 
system in which the sampling interval varies randomly over a fixed. 


2 


i 


Figure 2.3 



1. The Closed-Loop System Using Quasl-Steady-State 
Feedback. 
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finite set of positive numbers. The stability of the randomly sampled 
system can be analyzed by re-examining the discretization procedure for 
a continuous system. The optimal gaJ.ns calculated in section (2.3) are 
used to stabilize a linear system of the form 

X = Ax + Bu (2.4.1) 


If a finite set of positive real number {s^ * * * s^} is selected for 
possible sampling intervals and a constant u(k) is applied to produce 
the state x(k+l), s^ time units later, the state at the next sampling 
instant is given by 

A S . 

r ^ At 

x(k+l) - e ^ x(k) + e *^dt Bu(k) (2.4.2) 

Jo 

For the case in which the controller knows the duration of the 
sampling interval at its initiation, the constant u(k) is the product 
of the state and the gain matrix, which depends only on s^(the duration 
of the sampling interval), i.e. 


u(k) - F^x(k) (2.4.3) 

For the case in idiich the controller does not know the duration 
of the sampling interval at its initiation, there are possible combina- 
tions of gain and sampling intervals, but the basic approach will be 
the same. For simplicity we will stay with the first case. 


Introducing (2.4.3) into (2.4.2), 

s A 

x(k+l) - (e ^ + 


^e^^Bdt F^)x(k) 


(2.4.4) 


We define 
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“ e 


■i ■ 


t for i - 1, • N. 


(2.4.5) 


Equation (2.4.4) can now be written as 


x(fcfl) - + r^Fj_)x(k) 


(2.4.6) 


Hi - (»i + FiFi), 


(2.4.7) 


x(k+l) ■ H^x(k) 


(2.4.8) 


Since the sampling interval varies in time, we introduce the index 
ij^ to denote i— configuration at the sampling instant. Hence, 


x(k+l) - (♦ + r F )x(k) 


(2.4.9) 


where 1 £ i^ ^ N. 


H ® ♦ + r F 


(2.4.10) 


Using Equation (2.4.9), the state at the k^ sampling instant can be 


expressed as 


with 1 .1 i j Ji • 


x(k) = n H x(o) 
j=o 


The stability of the system (2.4.6) is, therefore, giiaranteed if 
the following sequence converges 


k-1 
n H, 
j-o "-j 


I' 

II 
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I 

I 

I 

I 

I 

I 

I 


5 


r 

1 


I 

I 

I 

I 


Before discussing the convergence of a set of matrices, let us 
define some definitions on vector and matrix norms, [29], [30]. 

Vector norms must satisfy the following relations : 

11*11 11*11 * 0 X ■ 0 

l|ax|| » |a| ||x|| for any scalar a 
ll**7|l i||*|| + ||y|| (2.4.U) 

Similarly, matrix norms have the following properties: 

1 |a| I ^ 0 and \\ a \\ - 0 iff A - 0 
I |oA| I » |a| I |a| I for any scalar a 
llA+Bil < IlAil + ||B|| 

llABll < ||a|1 1!b|1 (2.4.12) 


Corresponding to each vector norm, the induced matrix norm is 
defined as 

||a|| “max (2.4.13) 

It can be proved that | [a| | > max |Ax| ] 

||x| =1 

Three most commonly used vector norms are 

1 1*1 Ip “ cl*il^ + 1*2^ + I*nI^^^^^ 

for P - 1,2,®. (2.4.14) 

The norm | |x | (2 Is called the Euclidean norm and | |x| can be 

intern veted as maxjx. |. It Is conceivable that a measure of magnitude 
1 ’ ^ 

for matrices could be based on the magnitude of the eigenvalues. If 


•■SJ 


'i 


1 
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Ui ••• are the eigenvalues of A, then » max|u.| (1 < j ^ n) is 

j ^ 

called the spectral radius of A. The following theorem on spectral 
* 

radius of A A will he used to establish the stability conditions of the 
closed-loop system. 

Theorem 2.4.1 if j | • j | denote the matrix norm (known as spectral norm) 

X/2 

Induced by the Euclidean norm, then | |a| , where is the 

spectral radius of A^* A. 

ie 

Proof: Since the matrix A A is Hermitian and positive-definite, it is 

* 

well known that the eigenvalues of A A are real and non-negative. The 

•k 

spectral radius is, therefore, an eigenvalue of A A. 

Let {x^, X 2 » ***, x^} be a set of orthonormal right eigenvectors 

* 

of A A with associated eigenvalues A^, A^, *** A^. For any x with 

n 

I [x[ L » 1 write x ■ I 5.x., then 
^ j-1 J J 


* 

A Ax 


and 

(I 1ax| [ 2 )^ 


■k 

which uses the fact that x 


n 

Z 

j-1 




(Ax)*Ax 


it it 

X (A Ax) 


(2 


] VkV 


z 15.1% 
j J ^ 


6 . Therefore, 


llAxlIj - 


But 



^ 0 for j » 1, **’, n. 

It follows Immediately that 

Ikllg - maxllAxll2 = 

Since is chosen from the finite set {Hj^, *** We will 

state the definition of convergence of a set of matrices. 

Definition 2.4.1; A set H 2 • * * } of m x m matrices is conver 

gent if every sequence 

k 

{x, X, X, •**, ( n )x, 

o \ ^o j=o 

(for 1 £ ij £ N) converges to zero for every x in R^. 

The convergence can be written as 

k 

lim( n H )x “ 0 
fc+<n j«o j 

for all 1 < i. < N. 

“ J - 

It is well known that the single element set is convergent 

if and only if the spectral radius of is less than one — i.e. , 
satisfies the relation ||h^x |[2 < ||x|l 2 ^ where I |*||2 

denotes the Euclidean norm. One is tempted to conjecture that a set 
• • • Hjj} of m x m matrices is convergent if and only if every matrix 
H^, 1 £ i £ N has a spectral radius less than one. Unfortunately, 
this is not true. The following is a counterexample. 
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Example 2.4.1 Let the set of matrices be H 2 > where 


U, » , H„ = 

0 0 10 


It is clear that both and H 2 have spectral radii which equal 
zero. However, the following sequence will not contract the vector 


[0 1 ] 


Hj^H2H^H2 Hj^H2 ••• 


Therefore, the set of matrices {H^, H 2 } is not convergent. 

At present a necessary and sufficient condition for the conver- 
gence of a set of matrices is unknown. We have, however, established 
a sufficient condition for the convergence of a set of matrices. 

Theorem 2.4.2 A set H 2 , *•*, of m x m matrices is convergent 

if for every matrix H^, 1 £ i £ N, has a spectral radius less than 


Proof: from the definition of the spectral norm (the matrix norm in- 

duced by the Euclidean norm) we have 


\ Vl ^o ^k-1 ^o 


where 


1 £ ij £ N for j-0, •••, k 


H 

^k ^k-1 


1 MH^ 


wr“ 
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from Theorem 2.4.1, where X is the spectral radius of H H. . 

J j j 

Since every matrix has a spectral radius ^ess than one 

for 1 ^ i N and 1 £_ i. ^ N, therefore |X. j <1 for j “0, •*' , k. 

^ j 

Hence, each number in the sequence X . X . • • • X^ has an absolute 

K K-1 o 

value less than one, and 

k 

lim n X * 0 
-K» j-o j 

Therefore, the sequence of matrices will also converge. 

To verify Theorem 2.4.2, let us return to Example 2.4.1. It is 
* * 

obvious that both and H 2 H 2 have a spectral radius equal to 1. 

Therefore, the set of matrices {Hj^, H 2 } is not necessarily convergent 
under Theorem 2.4.2. 

From the above discussion, we can see that the convergence of 
the set of matrices **•» N} is a sufficient condi- 

tion for the stability of the randomly-sampled system. Although the 
condition looks very restrictive, it is required for reliable operation 
of the control system. Namely, if the set of matrices { (4^^ + , 

i-1, N} is convergent, the system will remain stable regardless of 

what the actual transition probability between adjacent sampling inter- 
vals is, since it is conceivable that the actual transition probability 
could change due to software or hardware failure in a digital system. 
The convergence is, however, not a necessary condition for finding a 
stable sequence. We will state the example from Reference [20] to show 
why it is not a necessary condition. The example is interesting be- 
cause it shows that for any given vector it is possible to find a 
stable sequence of matrices from a set of matrices in which each 
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element Is unstable by Itself. 


Example 2.4.2 Let the set of matrices be 


where 



It can be easily verified that each matrix has a spectral radius of 2. 

The set of matrices is not convergent by Theorem 2.4.2 and Definition 

2 

2.4.1. But for every vector in R it is always possible to find a 

sequence of matrices from the set which will contract the vector to 

zero for the following reasons: contracts every vector in the 

closed cone R^, co-axi^ll with the x axis, having 45* vertex angle at 

the origin. Similarly H 2 * and contract vectors in the cones R 2 > 

R^ and R^ which are counter-clockwise rotations of Rj^ through 45*, 

90® and 135®, respectively. Apparently, Rj^, R 2 , R^ and R^ cover the 

2 

whole region of interest. Hence, we can contract every vector in R 
using the set {H^, H 2 * The above example shows that while 

the convergence of the set of matrices guarantees the stability of the 
closed-loop system, it is not a necessary condition to find a stabiliz- 
ing sequence of matrices. 

To summarize, we state the following theorem for the stability 
of our closed-loop system. 
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Theorem 2.4.4 If each spectral radius of the matrices In the set 

ic 

{(♦^ + + r^Fj),l»l, *•*, N} is less than unity, the closed- 

loop optimal control system with random-sampling is stable regardless of 
what the actual set of transition probabilities between the adjacent 
sampling Intervals is. 

2. 5 The Existence of Quasi-Steady-State Solutions 

Optimal control problems with an infinite optimization interval 
are always of special Interest. The resulting control strategy, if it 
exists, is usually simple to implement, and with suitable assumptions 
yields a stable closed-loop system. The single sample-rate discrete- 
time system is said to be stabillzable if there exists a matrix F such 
that all the eigenvalues of (9 + FF) are inside the unit circle [2] , 
[28]. It is obvious that if a system is stabllzable, there always 
exists a control input i^lch makes the performance index with in- 
finite optimization interval finite. For the randomly-sampled system 
we state the following theorem on stablizability. 

Theorem 2.5.1 A randomly-sampled system with NS possible configura- 
tions r^), i»l, •••, NS> is stabillzable if for each pair 

(♦f, there is an such that the set of matrices {(4^^ + r^F^) i»l, 
•••, NS} is convergent. 

From Theorem 2.4.4 the following theorem is immediate. 

Theorem 2.5.2 If for each pair F^^) in the set F^^), i«l, •••, 

NS} of randomly- samp led system there exists an F^ such that all the 

T 

eigenvalues of I4>^ + + r^^F^] are inside the unit circle. 


Che system Is stabillzable. 

In order Co establish Che exisCeace of an optimal solution as 
the time index k -► we must prove certain properties of the matrices 
P(k/i), see (2.3.48), i-1, NS. Specifically, we will show that 

the sequence is bounded above and is non-decreasing. It is clear Chat 
P(k/i) is bounded if the system is stabilizable. The inequality 
P(k+l/i) > P(k/i) can be established if we have x^P(k+l/i)x > x^P(k/i)x 
We will prove x"^P(k+l/i)x > x^P(k/i)x as follows: 

Since each term in the cost function 

N-1 

J » y E{x(N)'^Sx(N) + E [x(k)'*^Q(k)x(k) + v'^(k)R(k)V(k) 1 } 

^ k-0 

is nonnegative, we have 

J*(N-k, N+1) ^J*(N-k, N). 

From equation (2.3.48), we have 

J*(N-k, N) » x^(N-k)P(k/i)x(N-k). 

It should be noted that the value of P(k/i) in equation (2.3.48) 
depends only on the number of iterations. The time invariance of the 
statistics of the system allows a shift from the interval (N-(k+l) , N) 
to (N-k, N+1). Thus, we have 

J(N - (k+1), N) - J(N-k, N+1) 

Therefore, 

P(k+l/i) i P(k/i) 

***This completP.fi the proof. The quasi-steady state can, of course 
be calculated by substituting P(k) for P(k-l) in equation (2.3.48) and 
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solving the resulting algebraic Rlccati equations. To sumnarlze, we 
state the following theorem: 


Theorem 2.5.3 If a linear system with random sampling is stabllizable, 

the quasi-steady state solution for the probabilistic Rlccati Equations 

(2.3.48) and (2.3.49) exist and can be determined from the following 

algebraic Rlccati equations. 

NS - 

P(k/i) - E T {QOl-k, j) + K^(N-k/j)R(N-k, j) 

j-1 ^ 

• K(N-k. j) + [^(N-k, j) + r(N-k, j)K(N-k/j)]^ 

♦ P(k/j)[4(N-k, j) + r(N-k, j) * K(M-k/j)l 

(2.5.1) 


K(N-k/i) 


- Z T. jR(N-k, j) + r'^(N-k, j) 
j-1 

• P(k-l/j)r(N-k, j)]"^H Z T. [r(N-k, m) 


• P(k-l/m)$(N-k, m)l> 


(2.5.2) 


Considerable computational time may be saved by using equations 
(2. 5. 1-2. 5. 2) Instead of eqiiatlons (2.3.48-2.3.49). If Theorem 2.5.3 
falls to Indicate the stabillzablllty of the system, equations 
(2.3.48-2.3.49) can be Iterated recursively, backward in time until 
either they converge or pass some test of non-convergence. Since the 
P(k/i) matrix is non-decreasing, it is not possible to have limit 
cycles. 
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3. APPLICATION TO AIRCRAFT CONTROL 


3.1 Introduction 

In this chapter we Illustrate the application of the optimal 
solution of Chapter 2 to the longitudinal control of the F8-DFBN 
(Dlgltal-Fly^By-Ulre) aircraft. The longlttidlnal dynamics of the 
F8-DFBW Is modeled as a linear process and the information distribu- 
tion process is modeled as a variable time-increment process where, 
at the time chat Information Is supplied to the control effectors, 
the control effector knows the time of the next Information update 
only in a stochastic sense. Section (3.2) describes the longitudinal 
dynamics of the F8-DFBW aircraft. The unaugmented response of the 
aircraft to Initial conditions Indicates the need for control. 

In section (3.3) the aircraft control problem Is modeled as an 
optimal control problem with random sampling. The optimal control 
problem for the aircraft is solved in section (3.4). Theoretical 
and simulation results Indicate that the optimal cost obtained using 
a variable time-increment Markov Information update process, l.e. 
where the control effectors know only the past information update 
Intervals and the Markov transition mechanism. Is almost identical 
to that obtained using a known uniform information update Interval. 

3.2 The Equations of Motion 

The longitudinal equations of motion for a rigid aircraft are: 
mv * ZF^ ■ T cos a - mg sin (9 - a) - D (3.2.1) 
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mva ■ nivq - T sin a + mg cos (0 ~ a) - L (3.2.2) 

I q - ZM - M + M (3.2.3) 

y y 3 t 

0 ■ q (3.2.4) 

where 

V • forward velocity 
a » angle of attack 
q » pitch rate 
6 « pitch angle 

T » thrust, D “ drag, L * 11 :t 

*■ pitching moment due to aerodynamic forces 
Mj. ■ 0 ■ pitching moment due to thrust 
m ■ mass, g ■ gravity, 1^ ■ moment of inertia 
about the y-axls. 

The state variables of the aircraft longitudinal system are 
forward velocity, angle of attack a, pitch rate q and pitch angle 6. 
The aerodynamic forces and moments are the lift and drag forces and 
the pitching moment. These forces and moments can be expressed in 
terms of aerodynamic data as follows: 

D - i pv^SCjj (u, o, a, q, 6e, of) (3.2.5) 

L « pv SCj^ (u, o, o, q, 6e, 6f) (3.2.6) 

12 ^ * 

M ■ y pv ScC^ (u, o, a, q, fie, fif) 


(3.2.7) 




S ” the effective aerodynamic surface area 
p “ atmosphere density 
c > the mean aerodynamic wing chord 

, « nonlinear drag, lift, and moment coefficients. 

( 1 , 5e, 5f * perturbation in velocity and elevator and flap 
surface deflections 



For a nonlinear dynamic system 

i-f(5F, u) (3.2.8) 

The small perturbations around the nominal value (x , IT ) can 

o o 

be expressed as 

X ■ X + Tx (3.2.9) 

o 

“ H + Tu (3.2.10) 

o o 

Substituting equations (3.2.9) and (3.2.10) Into (3.2.8). we 

have 


X + ^ ■ f (x + 5x, y + Tu) (3.2.11) 

O 0 0 

Taking a Taylor series expansion, we have 

? + 6x • f (jc , IT ) + 1“ (x , TT ) 6x 

O O O dX O O 

3f 

+ — (y , TT ) 6y + Higher order terms 


(3.2.12) 



Neglecting higher order terms, we have the following linearized 


equation. 


6x ■ A5x + B61T 


where 


- i f'o* V 


» - If V 


For the F8-DFBW aircraft the linearized eqiiatlons of 
at a selected trimmed flight condition have the matrix form 


V 


• 

a 

1 

q 


_e 

(_ 


1 

> 

2 

3 


12 



V 


o 

1 


a 


^8 *10 


q 


*9 *11 


_e 


I 

o 


6e 

5f 


where 


P ■ - C 
1 mV Du 
o 


-^c 

mV 2 


P . £§c_ . _s^ 

‘'s VI ^ mu - ,, 

o y 2mVr 


7 C • C, ) 
2 ma Lu 


(3.2.13) 

(3.2.14) 

(3.2.15) 
motion 

(3.2.16) 


57.3qS ^ 
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_ 57.3^ 

5 mV 

o 




P - 57»3gsc (c _ 3§^ c • c . ) 

7 ly m6e 2 V ^ 

o 

^ _ 57 . 3 ^ 




.ILias 

9 mV ^6e 


p .57^ 

10 m ^6f 


p . 57y3lS 

11 mV ^6f 
o 


3ST 


12 2V I ^^mq ■‘‘ ^ma^ 
o y ^ 


- 1 2 
q .-pVo- 


For the equilibrium flight condition at an altitude of 20,000 
feet and a Mach number of 0.67, the linearized longitudinal dynamics 
take the following numerical values 


A 


-9.529 E - 3 
-1.175 E - 4 
3.324 E - 7 
0 


-1.283 E + 1 
-9.782 E - 1 
-4.723 E + 0 
0 


0 

1.0 

-4.729 E - 1 

1.0 


-3.217 E + 1 
0 
0 
0 
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-6.554 E + 0 
-2.253 E - 1 
-1.539 E + 0 
0 


-1.513 E - 1 
-1.333 E + 0 
0 


3.3 Formulation of a Randomly-Sampled Aircraft Control Problem 
For a "staticly stable aircraft" all eigenvalues of the A 
matrix must have negative real parts. A staticly stable aircraft 
will return to its equilibrium position after a small disturbance 
has occurred. Although all aircraft are designed to be staticly 
stable, certain disturbances may result in unsatisfactory responses. 
Flg\jre (3.3.1) shows the unaugmented response of the aircraft to 
initial conditions. The motion is characterized by two oscillatory 
modes, one of short period and one of long period (phugoid). The 
predominant oscillation, seen in Figure (3.3.1), is the short-period 
oscillation which has a period of 3 seconds. The large variations of 
the angle of attack and the pitch angle are not acceptable. In 
addition, the short-period mode causes the aircraft to have a pro- 
longed oscillation. The main control objective is to damp the short- 
period mode. The weighting matrices, Q and R, of Equation (2.2.6), 
were selected so that only elevator deflections are used to control the 
aircraft and so that only the short-period mode is regulated. This 
requires high weights on the a- and 0-error terms and a high weight 
on the flap cost term. The v and 9 weighting terms are small so as 
not to over-control the long-period mode. 
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Errors in the state variables are weighted relative to control 
effort by adjusting the Q and R matrices in the cost functional. The 
specified values for the Q and R matrices are 


2.5 E - 5 
0 
0 
0 


3.0 E + 0 
0 
0 


0 0 

0 0 

3.0 E + 0 0 

0 1 . 


TT — 

D a.! 


R 


4.0 E + 0 0 

_0 4.0 E - 1 


The sample space s chosen to be t * {0.02, 0.03, 0.1} and 
the transition probability is chosen to be 


T 


95 . 05 o' 

0 .8 .2 

.7 0 .3 


Hence, if the sampling interval at t^_^ is the probability 
that is 0.95, the probability that t^ » $2 is 0.05, and the 

probability that t^ ■ is zero. Figure (3.3.2) is a graphical 
representation of transition mechanism. A characteristic of this 
transition mechanism is that it leads to a process which is cyclic. 
That is, it tends to chang>^ gradually from the fastest sampling 
interval to the slowest and back to the fastest. Table (3.3.1) 



Table 3.3 


Sequence 1 = 
Sequence 2 = 




AS 


. 1 Sequences generated by Random Number Generator 








= 2 * 


Sequence 3 = {S^, S^^, S^^, S^, S^, S^, Sj , S^, S^^, S^, S^, S^, S^} 
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shows several sampling sequences generated by a random number 
generator using the indicated transition probability. 

3.4 Numerical Results and Conclusions 

In this section some numerical results on the F8-DFBW aircraft 
longitudinal system with random sampling will be presented. For the 
chosen sample space t * {0.02, 0.03, 0.1}, the equivalent discrete 
model is obtained for each sampling interval using equations (2.2.13- 
2.2.18) and the software in Reference [19]. The results are sho;m 
in Tables (3.4.1), (3.4.2), and (3.4.3). For the purpose of comparison, 
the optimal control system with uniform sampling is solved for each 
sampling interval. The optimal gains for the uniformly-sampled 
system were first obtained by solving the stochastic Rlccatl equations 
(2.3.47 and 2.3.48) Iteratively with the transition probability 
matrix T « I. In this approach three steady-state gains for the 
system with uniform sampling intervals S^, and were obtained 
simultaneously. Each gain matrix was verified by solving the regular 
Riccati equation individually using the software in Reference [19] . 

The results were identical. The steady-state solutions of the 
uniformly-sampled system are shown on Tables (3. 4. 4-3. 4 .6) . The 
uniformly-sampled system was simulated for each sampling interval 
using the corresponding optimal gain matrix. The optimal responses 
for each system are shown in Figures (3. 4. 1-3. 4. 3) - It should be 
noted that the response Improves as the time interval becomes smaller. 

It should be emphasized once more that the uniform.! ' -sampled systems 
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are not realizable by our assumption and the systems are simulated 
here only for the purpose of comparison. The optimal gains for the 
randomly-sampled cases were also obtained by solving the stochastic 
Riccati equations (2.3.47 and 2.3.43) iteratively with the transition 
probability T as given in Figure (3.3.2). In this case the optimal 
control law consists of three quasi-steady state gain matrices 
** K(»/i) with i ■ 1, 2, 3. If the sampling interval at the 
(k - l)-th interval t^^ ^ is S^, the control u(k) applied over the 
k-th time interval t^ is given by u(k) = K^x(k). The quasi-steady- 
state gain matrices and sensitivity matrices are shown in Tables 
(3.4.8) and (3.4.7), respectively. It is interesting to see that 

the norms of the sensitivity matrices are very close to each other. 

T 

Therefore, the expected cost, which is equal to x (0) P(“/i) x(0) , 
is almost the same regardless which initial sampling interval t^ is 
assumed. The quasi-steady-state gain matrices, on the other hand, 
are considerably different from each other. Hence, it is Important 
to use the correct gain for each sampling interval. 

A randomly-sampled system is simulated with the transition 
probability between the adjacent sampling intervals equal to the 
matrix shown in Figure (3.2.2). The quasi-steady gain from the 
optimal stochastic regulators is first applied to the randomly- 
sampled system. The response of the system is probabilistic, i.e. each 
run is different from the other even if the initial state is the same. 
Figure (3.4.4) shows a typical response of the randomly-sampled 
system using optimal control. Comparing these results with the 
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deterministic optimal solution with known uniform sampling intervals 

in Figure (3.4.1), we can see that the trajectories differ only 

insignificantly. The expected value of the cost of the process can 

also be evaluated using the sensitivity matrices according to the 
T 

formular x (0) P(«/i) x(0). That value is 0.5096 for uniform 
sampling (.02 sec/sample) and 0.5159 for stochastic sampling. This 
indicates that little is lost in the stochastic information update 
process provided that the stochastic nature of the process is 
accounted for in the control logic. The closeness of the stochastic 
and deterministic trajectories are remarkable since the determinis- 
tic optimal control requires absolute knowledge of the whole sampling 
sequence, which, of coiurse, is not realizable for the problem we are 
considering. Non-optimal gains are applied to the randomly-sampled 
system to study the sensitivity of the feedback gains. Figures 
(3.4.5), (3.4.6) and (3.4.7) Illustrate typical responses of the 
randomly-sampled systems using "optimal gains" calculated for a 
uniformly-sampled system with sampling Interval equal to 0.02 sec., 
0.03 sec. and 0.1 sec., respectively. To study the average cost, 

50 simulations were performed for the optimal feedback system and 
each non-optimal feedback system. The results are summarized in 
Table (3.4.9). They are consistent with our theory, i.e. in the 
randomly-sampled system the optimal stochastic gains give the lowest 
average cost (0.5174) over all other gains, including those calcu- 
lated for the uniformly-sampled system. 
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Table 3.4.1. 

Discrete Model 

for Sampling Interval » .02 Sec. 


Phi Matrix 

4 Rows 

4 Columns 


9.9980E-01 

-2.5395E-01 

-8.9546E-03 

-6.4333E-01 

-2.3265E-06 

9.7969E-01 

1.9705E-02 

7.5103E-07 

1.1653E-07 

-9.3074E-02 

9.8965E-01 

-2.5765E-08 

8.0091E-10 

-9.3540E-04 

1.9899E-02 

l.OOOOE+00 

Gam Matrix 

4 Rows 

2 Columns 


-1.3040E-01 

1.1827E-03 



-4.7669E-03 

-5.6363E-03 



-3.0430E-02 

-2.6517E-01 



-3.0553E-04 

-2.6568E-03 



Q Matrix 

4 Rows 

4 Columns 


4.9990E-07 

-1.3299E-07 

-3.2405E-11 

-1.6081E-07 

-1.3299E^07 

5.8979E-02 

-2.2011E-03 

-2.0356E-08 

-3.2405E-11 

-2.2011E-03 

5.9406E-02 

1.9939E-06 

-1.6081E-07 

-2.0356E-08 

1.9939E-06 

2.0006E-04 

W Matrix 

4 Rows 

2 Columns 


-6.4985E-08 

-1.5411E-10 



-1.6300E-04 

7 . 1016E-04 



-1.8245E-03 

-1.5837E-02 



-1.2901E-08 

-3.5469E-07 



R Matrix 

2 Rows 

2 Columns 


8.0019E-02 

1.6258E-04 




1.6258E-04 


9.4109E-03 
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Table 3.4.2. 

Discrete Model 

for Sampling Interval ■ .03 Sec. 


Phi Matrix 

4 Rows 

4 Columns 


9.9971E-01 

-3.7854E-01 

-2.0093E-02 

-9.6496E-01 

-3.4712E-06 

9.6900E-01 

2.9333E-02 

1.6839E-06 

2.5597E-07 

-1.3854E-01 

9.8382E-01 

-8.4251E-08 

2.6189E-09 

-2.0941E-03 

2.9767E-02 

l.OOOOE+00 

Gam Matrix 

4 Rows 

2 Columns 


-1.9500E-01 

3.5489E-03 



-7.3405E-03 

-1.0381E-02 



-4.5363E-02 

-3.9656E-01 



-6.8466E-04 

-5.9662E-03 



Q Matrix 

4 Rows 

4 Columns 


7.4978E-07 

-2.9745E-07 

-3.5255E-10 

-3.6180E-07 

-2.9745E-07 

8.7869E-02 

-4.9137E-03 

-6.8748E-08 

-3.5255E-10 

-4.9137E-03 

8.8637E-02 

4.4800E-06 

3.6180E-07 

-6.8748E-08 

4.4800E-06 

3.0023E-04 

W Matrix 

4 Rows 

2 Columns 


-1.4569E-07 

-1.3404E-09 



-2.4918E-04 

2.5790E-03 



-4.0778E-03 

-3.5439E-02 



-4.3452E-06 

-1.1955E-06 



R Matrix 

2 Rows 

2 Columns 


1.2006E-01 

5.4536E-04 



5.4536E-04 

1.6741E-02 




Table 3.4.3. Discrete Model for Sampling Interval 


10 Sec 



Phi Matrix 

4 Rows 

4 Columns 


9.9905E-01 

-1.1885E+00 

-2.1859E-01 

-3.2154E+00 

-l.llOlE-05 

8.8512E-01 

9.2281E-02 

1.8223E-05 

2.6658E-06 

-4.3586E-01 

9.3174E-01 

-2.9155E-06 

9.0630E>08 

-2.2417E-02 

9.6923E-02 

9.9999E-01 

Gam Matrix 

4 Rows 

2 Columns 


-6.3000E-01 

1.0718E-01 



-2.8607E-02 

-7.7585E-02 



-1.4419E-01 

-1.2888E+00 



-7.3792E-03 

-6.5259E-02 



Q Matrix 

4 Rows 

4 Columns 


2.4976E-06 

-3.1953E-06 

-3.0913E-08 

-4.0174E-06 

-3.1953E-06 

2.8821E-01 

-5.1246E-02 

-2.5491E-06 

-3.0913E-08 

-5.1246E-02 

2.8283E-01 

4.9393E-05 

-4.0174E-06 

-2.5491E-06 

4.9393E-05 

1.0086E-03 

W Matrix 

4 Rows 

2 Columns 


-1.5976E-06 

-2.4818E-07 



5.6752E-03 

1.0009E-01 



-4.2932E-02 

-3.7628E-01 



-1.5797E-06 

-4.3838E-05 



R Matrix 

2 Rows 

2 Columns 


4.0222E-01 

1.9270E-02 



1.9270E-02 

2.0976E-01 
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Table 3.4.4. Optimal Solution for Uniformly Sampled 
System with Sampling Interval ■ .02 Sec. 



Gain for Tau (1) ■ . 

20000E-01 


G(l) Matrix 

2 Rows 

4 Columns 


1.8535E-04 

7.8963E-02 

2.2683E-02 

-2.5638E-02 

-5.2238E-03 

3.1285E-01 

1.9756E+00 

8.8219E-01 

Sensitivity Matrix for Tau 

(3) - .lOOOOE+00 


P Matrix 

4 Rows 

4 Columns 


1.1994E-04 

1.0675E-03 

-3.3902E-04 

-9.4343E-03 

1.0675E-03 

1.4353E+00 

1.3888E-02 

-3.9375E-01 

-3.3902E-04 

1.3888E-02 

1.1996E-01 

5.8485E-02 


-9.4343E-03 


-3.9375E-01 


5.8485E-02 


1.7723E+00 
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Table 3.4.5. Optimal Solution for Uniformly Sampled 
System with Sampling Interval « .03 Sec. 



Gain for Tau (2) 

- .30000E-01 


G(l) Matrix 

2 Rows 

4 Columns 


1.9892E-04 

7.7499E-02 

1.9770E-02 

-2.7282E-02 

-4.6552E-03 

2.2307E-01 

1.7167E+00 

7.7317E-01 

Sensitivity Matrix for 

Tau (2) » .30000E-01 


P Matrix 

4 Rows 

4 Columns 


1.1922E-04 

1.0879E-03 

-2.4621E-04 

-9.3346E-03 

1.0879E-03 

1.4311E+00 

2.5550E-03 

-3.9702E-01 

-2.4621E-04 

2.5550E-03 

8.6224E-02 

4.3228E-02 


-9.3346E-03 


-3.9702E-01 


4.3228E-02 


1.7566E+00 
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Table 3.4.6. Optimal Solution for Uniformly Sampled 
System with Sampling Interval » .10 Sec. 

Gain for Tau (3) ■ .lOOOOE+00 
G(l) Matrix 2 Rows 4 Columns 

2.2852E-04 7.1368E-02 1.0267E-02 -3.2493E-02 

-2.2871E-03 -7.7732E-02 8.3336E-01 3.8282E-01 

Sensitivity Matrix for Tau (1) ■ .20000E-01 
P Matrix ^ 4 Rows 4 Columns 

1.1354E-04 1.1378E-03 -2.3509E-04 -9.1732E-03 

1.1378E-03 1.4297E+00 1.7943E-03 -3.9664E-01 

-2.3509E-04 1.7943E-03 8.4000E-02 4.1950E-02 


-9.1732E-03 


-3.9664E-01 


4.1950E-02 


1.7453E+00 






ALPHA, RAD 


3. RAO/SEC 


THETA. RAD 


OF. RAD 


DE, RAD 







V- 
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Table 3.4.7. Quasi-Steady State Sensitivity Matrix of 
Stochastic Ricatti Equations. 



Sensitivity Matrix for 

Tau 

(1) = .20000E-01 


P Matrix 

4 Rows 


4 Columns 


1.1978E-04 

1.1080E-03 


-2.4544E-04 

-9.4068E-03 

1.1080E-03 

1.4327E+00 


1.8046E-03 

-4.0116E-01 

-2.4544E-04 

1.8046E-03 


8.5011E-02 

4.3119E-02 

-9.4068E-03 

-4.0116E-01 


4.3119E-02 

1.7696E+00 

Sensitivity Matrix for 

Tau 

(2) = .30000E-01 


P Matrix 

4 Rows 


4 Columns 


1.1996E-04 

1.0959E-03 


-3.0399E-04 

-9.4375E-03 

1.0959E-03 

1.4343E+00 


6.2613E-03 

-3.9918E-01 

-3.0399E-04 

6.2613E-03 


1.0405E-01 

5.2773E-02 

-9.4375E-03 

-3.9918E-01 


5.2773E-02 

1.7747E+00 

Sensitivity Matrix for 

Tau 

(3) = .lOOOOE+00 


P Matrix 

4 Rows 


4 Columns 


1.1985E-04 

1.0977E-03 


-2.7469E-04 

-9.4191E-03 

1.097 7 E-03 

1.4338E+00 


5.4788E-03 

-3.994E-01 

-2.7469E-04 

5.4788E-03 


9.6078E-02 

4.7938E-02 

-9.4191E-03 

-3.9947E-01 


4.7938E-02 

1.7717E+00 
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Table 3.4.8. Quasi-Steady State Gain Matrix of Stochastic 
Riccati Equations . 




Gain for Tau (2) = 

.30000E-01 


G(l) Matrix 

2 Rows 

4 Columns 


2.1973E-04 

7.3006E-02 

1.4131E-02 

-3.0984E-02 

-2.9917E-03 

1 . 2004E-01 

1.2040E+00 

4.9917E-01 


Gain for Tau (1) = 

.20000E-01 


G(l) Matrix 

2 Rows 

4 Columns 


1.9223E-04 

7.8889E-02 

2.2274E-02 

-2.6143E-02 

-5.3798E-03 

3.0465E-01 

1.9414E+00 

8.9281E-01 


Gain for Tau (3) = 

.lOOOOE+00 


G(l) Matrix 

2 Rows 

4 Columns 


2.2576E-04 

7.4A35E-02 

1.1344E-02 

-3.1862E-02 


-2.5799E-03 


-5.9389E-02 


9.4410E-01 


4.3141E-01 
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Figure 3.4.5. 




Response of Randomly- Samp led System Using 
Optimal Gain Calculac'-j for Uniformly-Sampled 
System with Sampling Interval « .02 Sec. 
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Figure 3.4.6. Response of Randomly- Samp led System Using 

Optimal Gains Calculated for Uniformly-Sampled 
System with Sampling Interval ■ .03 Sec. 
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Figure 3.4.7. Response of Randomly-Gempled System Using 

Optimal Cains Calculated for Uniformly- Sampled 
System with Sampling Interval • .10 Sec. 



Table 3.4.9. Results of Simulation Study: Averages of 50 

Random Runs of 10-Second Durations 


Gains 

Used 

Optimal Gain 
for a 0.02 sec. 
Uniformly- 
Sampled System 

Optimal Gain 
for a 0.03 sec. 
Uniformly- 
Sampled System 

Optimal Gain 
for a 0.10 sec. 
Uniformly- 
Sampled System 

Unaugmented 

System 

System with 
Optimal 
Stochastic 
Gains 

Average 

Cost 

0.6158 

0.5526 

0.5372 



1.427 

0.5174 


4. RANDOMLY-SAMPLED STOCHASTIC SYSTEM CONTROL 


4.1 Introduction 

In Chapter (2) optimal control of linear systems with 
stochastic sampling was discussed. A more practical concern Is 
with a more general stochastic control problem. Specifically, we 
focus our attention on a randomly-sampled linear process model In 
which additive process and measurement noise are present. The 
control effectors In the process have variable sampling intervals 
between Information updates. The Information update Intervals are 
modeled as an Independent stochastic process with known transition 
probabilities relating the duration of adjacent intervals. We will 
mlniml.'^e the mathematical expectation of a quadratic form in the state 
variables and control variables. As In the optimal control of 
stochastic systems with uniform sampling, the determination of the 
optimal strategy involves two problems, l.e. the problem of optimal 
control and the problem of optimal estimation. On the standard 
deterministic linear-quadratic-Gaussian (LQG) problem the separation 
theorem for the control and estimation holds [3], [4]. In this case 
the stochastic optimal feedback controller can be synthesized by 
cascading an optimal estimation (Kalman-Bucy filter) with the deter- 
ministic optimal control [21], [22]. Unfortunately, in most 
stochastic control systems with multiplicative noise, the optimal 
solution does not separate in the sense that the filter gains are 
not independent of the control computation [23], [24], [25], [26]. 
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In other words, the optimal control has to perform simultaneously 
the estimation and control of the state in order to minimize the 
expected cost of some real-valued cost function. 

For the stochastic control system with random sampling, the 
optimal estimator is equivalent to the time-varying Kalman filter 
because the estimator has perfect recall of the previous sampling 
intervals. Furthermore, the separation theorem still holds. The 
separation of estimation and control is Important because it con- 
siderably simplifies the implementation of the stochastic optimal 
controller for randomly-sampled systems [4], [27 . Tn section (4.2) 
we discussed the formulation of the stochastic control system with 
random sampling. Section (4.3) solves the optimal estimation and 
one-stage prediction problems for randomly-sampled stochastic 
systems. Section (4.4) presents the Separation Theorem, and section 
(4.5) provides some simulation results. 


4.2 The Formulation of the Stochastic Control Problem 

In a manner similar to our treatment of the deterministic 
system with random sampling, the control effectors are assumed to 
receive state Information at time for k >■ 1, 2, 3, . . . and the 
control actions are subject to the constraint u(o) ■ 

^ c ^ ®k+l . . . The sequence is random and 

can be described by the incremental process (t^} with an initial 
value 0 ^, so that 
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I 

8 

I 

I 


k 

0 . » a + y t. k ■ 1, 2, 3, . . . (4.2.1) 

° j-1 J 

with {t|^} a stochastic process. The systems we are considering are 
governed by the following stochastic difference equations 

x(k + 1) ■ *(k + 1, k) x(k) + ^Kk + 1, k) w(k) 

r(k + 1, k) u(k) (4.2.2) 

with the measurement equations 

y(k + 1) - Hx(k + 1) + 0(k + 1) (4.2.3) 

where w(k) and 9(k + 1) are mutually Independent, zero-mean Gaussian 
white noises with known statistics given by 


E{w(k), 

w(k)} - a(k) 

(4.2.4) 

E{0(k), 

w(k)} • 0 

(4.2.5) 

E{0(k), 

0(k)} - 6(k) 

(4.2.6) 


It should be pointed out that (w(k)}, {0(k)} and {t^} are mutually 
Independent processes, but that the matrices $(k + 1, k), T(k + 1, k) 
and <{j(k + 1, k) depend on tj^. It is also assumed that (w(k)}, 

{0(k)> and {t^} are Independent of x(N) and the initial state x(0), 
which has a normal distribution with 

E{x(0)} - m (4.2.7) 

Cov (x(0), x(0)} - Y(0) (4.2.8) 

The optimal stochastic control problem is to determine a 
closed-loop control strategy based on the past and current 
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measurements, past control and past sampling history to minimize the 
mathematical expectation of a quadratic function of the state and 
control variables. 

N 

V(N) - Min E{ I x (i) Q(i) x(i) 

u(0), . . . u(N - 1) 

+ xx^(± - 1) R(i - 1) u(i - 1)} 

(4.2.9) 

subject to the dynamics of Equation (4.2.2) and the measurements of 
Equation (4,2.3). The weighting matrices Q(i) are assumed to be 
positive semi-definite and the matrices R(i), positive definite. 
Similar to the deterministic control problem in Chapter (2) , we 
denote the entire sampling history by T(k) 

T(k) - (tQ, tj^, . . . , y (4.2.10) 

where T(k) e T(k). 

In analogy to T(k), the vector Y(k) is introduced here to 
describe the entire measurement history 

Y(k) - (y(o) y(k)} 2.11) 


where Y (k) e i/(k) . 

4.3 0pt^Tnal Esf»"«ation for Stochastic Systems with Random Sampling 
The optimal estimation problem is to obtain a best estimate 
ii(k) for x(k) given the measurement history Y(k) and sampling history 
T(k). For the system we have been considering if T(k) is given, 
all system matrices ♦(t), 4'(t) and F(t) are known for t - 1, ...» k. 
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The optimal estimator Is, therefore, same as the standard Kalman 
filter up to t ■ k, which satisfies the recursive relations stated 
in the following theorem: 

Theorem 4.3.1 For the stochastic system with random sampling, the 
optimally filtered estimate x(k + 1/k + 1) is given by the recursive 
relation 

x(k + 1/k + 1) - *(k + 1, k) x(k/k) + F(k + 1. k) u(k) 

+ K(k + l)[y(k + 1) - H*(k + 1, k) ii(k/k)] 

(4.3.1) 

where the gain matrix is determined by the following recursive 
relations 

K(k) - $(k + 1, k) P (k) H^[HP(k) + a(k)]"^ (4.3.2) 

P(k + 1) - [*(k + 1, k) - K(k) H] P(k) [*(k + 1, k) 

- K(k) H]^ + S(k) + K(k) a(k) K(k)^ 

for k - 1, 2, . . . (4.3.3) 

where P(k) is the covariance of the vector estimation error. The 
initial conditions are 


P(0) - y(0) 
x(0) • m 

The filter error {x(k + 1/k +1)} defined as 

x(k + 1/k + 1) - x(k + 1) - x(k + 1/k + 1) 


for k » 0, 1, . . . is a zero-mean Gtuss-Markov sequence whose 
covariance is given by (4.3.3). 

The Kalman filter (4. 3. 1-4. 3. 3' for the stochastic system 
with random sampling is, however, time-variant because the system 
matrices switch randomly among M possible configurations. In 
general, the gain matrix K(k + 1) is not expected to reach any type 
of steady-state. But there is no need to store any measurement data 
since the measurements can be processed on line as they occur. 
x(k/k) is the only estimate of state needed to be stored at time k. 
Since we will not have a steady-state gain, all possible configurations 
must be stored in order to calculate the optimal gain which is 
usually time-variant. Figure 4.3.1 gives the block diagram for the 
stochastic system with random sampling showing the Information 
flow inside the filter. It should be emphasized that the duration 
of the last interval along with the new measurement must be provided 
to the filter in order to proceed one step in time. 

The prediction problem for a stochastic system with random 
sampling is considerably complicated in that the predictor does not 
have exact knowledge of future sampling Intervals and instead knows 
only their probability distributions. For simplicity, let us assume 
the sampling rate switches randomly among m possible discrete values. 
The i~ of m linearized models is represented by and F^, 

the set of which is denoted by S^. Since the discrete-time system 
constantly switches among m models, S.(k) represents the ith model 
(configuration) at the kth sampling instant. Since the stochastic 
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sampling process is modeled as a semi-Markov process in which the 
adjacent sampling intervals switch according to the transition 
probability matrix, the probability of ith model at time k + 1 is 
conditioned on the last sampling interval value and represented by 


n.(k + 1) = Prob. {S.(k + l)/t, } 

1. X R 


For simplicity we will write the system model (4.2.2) in 
the following form 


x(k + 1) = '{’(k) x(k) + r(k) u(k) + i(j(k) w(k) 


(4.3.5) 


y(k) = Hx(k) + 0(k) 


(4.3.6) 


for k = 1, .... n 


$(k) e {$- $ } 

1 m 


ii)(k.) e {ij;. , . . . , li; } 
1 m 


r(k) £ {r, , . . . , r } 

± m 


Si A {$., li;., r^} 


The optimal predition of the state x(k 4- 1) at time k + 1 is 


x(k + 1) = E[x(k + l)/x(k)] 


(4.3.7) 


^C(k + 1) » / xp {x(k + 1) = x/x(k)} dx (4.3.8) 


Using S ^(k) to represent the ith configuration at time the 
conditional probability in Equation (4.3.6) can be written as 
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P(x(k + 1) = x/x(k)) 

M 

= I P(x(k + 1) = x/x(k), S.(k + D) n.(k + 1) (4.3.9) 

i=l ^ ^ 

Combining equations (4.3.9) and (4.3.4), we have 

m 

Y(k + 1) - / X I P(x(k + 1) - x/x(k), S (k +1)) dx n (k + 1) 
i-1 

m 

jc(k + 1) = ^ [/ xP(x(k + 1) “ x/x(k), S.(k + 1)) dx] n.(k + 1) 

i=l ^ ^ 

m 

x(k + 1) = ^ x.(k + 1) n.(k + 1) . (4.3.10) 

i=l ^ ^ 

Since x^(k + 1) can be obtained from the standard Kalman 
filter algorithm for predition, 

^^(k + 1) * x(k) + u(k) (4.3.11) 

Hence, Equation (4.3.10) becomes 

m 

x(k + 1) = y ($. x(k) + u(k)) n.(k + 1) (4.3.12) 

i=l ^ ^ ^ 

ir(k + 1) = T(k) x(k) + ^(k) u(k) (4.3.13 

where ^(k) and F(k) denote the expected values of the coriTesponding 
matrices conditioned on knowledge of previous sampling intervals. 
Equation (■♦.3.13) is used for one-stage prediction. 
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4.4 A Separation Theorem for Randomly -Sampled Systems 

In a manner analogous to our treatment of the randomly- 
sampled system without noise in Chapter 2, we will prove the Separa- 
tion Property for randor.ly-sampled systems by dynamic programming. 
First we consider a single-stage optimization problem over the last 

Interval which starts at . and terminates at and then we 

N-1 N 

generalize the result to all N stages by mathematical induction. 
Single-Stage Problem 

The optimal terminal cost for the single-stage process which 
starts at time N-1 and ends at time N is equal to 

V(l) - E{x’‘’(N) Q(N) x(N) + u''^(N - 1) 

. R(N - 1) u(N - 1)} (4.4.1) 

From the equation of our system model, we have 

x(N) - <t(N, N-1) x(N - 1) + '^(N, N-1) 

w(N - 1) + r(N, N - 1) u(N - 1) (4.4.2) 

Introducing equation (4.4.2) into (4.4.1). we have 

V(l) » E{($x + ’i»w + ru)’'^Q 

T 

($x + liiw + Tu) + u Ru} (4.4.3) 


where x = x(N - 1), w = w(N -1), u = u(N - 1) , 
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Q = Q(N), R - R(N - 1), 4 • J (N, N - 1), 

r * r(N, N - 1) and - 4(N, N - 1) . 

After some manipulation, we have 

V(l) » + Zx^J^QFu 

+ 2w^i|)'^Qru + ■{• u^CF^QF + R) u} (4.4.4) 

According to our assumptions, x(i), tf(i) and S(i + 1) are statis- 
tically independent of each other and w(i) has zero mean. Therefore, 
the expected value of the second, third and fourth terms in Equation 
(4.4.4) are zero. The optimal cost becomes 

V(l) - E (x^4^Q4x + 2x^4^QFu 

+ + u^(F^QF + R) u] . 

Using a property of conditional expectation; namely, that E {x} * 

E {E(x/y)}, where the outer expectation is over y, we have 

V(l) - E {E[x^4^Q4x + 2x^4^QFu + w\^Q-4jw 

+ u'^(fV + R) u/y(N - 1), . 

We will evaluate the second term of the above expression to show how 
the conditional expectation should be evaluated 


79 


E {2xVqru/y(N - 1), 

- E {2x^*^Qr/y(N - 1), u (4.4.5) 

m 

- [ [E {2x ^ qr/y(N - 1), Si(N) r] ••1^(N)*u (4.4.6) 

where 

•1^(k) « Prob ^S^(k)/tj^_j^> 

The expected value can be evaluated as the following 

E (2xVqr/y(N - 1), S^(N)} - 2x^^.q^r. (4.4.7) 

where the index i denotes the ith configuration Taking the sum- 
mation over i, we have 

E {2x^$^QFu/y(N - 1), t^_j} - 2x 5^qr*u (4.4.8) 

where $^qr denotes the expected value of $^qP given the value of the 
last sampling interval. Talcing the gradient of V(l) with respect 
to u and setting the result equal to zero, we have 

u - - (r^qr + r)~^ 't^qr x (4.4.9) 


It should be emphasized that x is the optimal estimate of x(N - 1) 
from the Kalman filter algorithm. Substituting the equation for u 
into the equation for V(l), 
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v(i) » E {x^$^Q«x - 2 x^t^Qr (r^qr + r)“^ r^qt x 

+ w^.{;^Q\i;w + x^ *^qr (r^qr f r)"^ r^q$ a} (4.4.10) 

After some manlpuletion, 

v(i) - E {x^ [♦^Q* - (r^qr + r)“^ r^q$l x 

+ E {x^ [4^qr (r^qr + r)‘^ r^q*l ii + w^/q-jw; (4.4.11) 

where x ■ x - x (4.4.12) 

Summarizing the results for the single-stage problem where P(N) * 
q(N) , we have 

u(N - 1) - S(N - 1) ii(N - l/y(N - 1), t^_^^) (4.4.13) 

where 

S(N - 1) « - [r’^(N, N - 1) P(N) r(N. N - 1) 

R(N - 1)]‘^ r^(N. N - 1) P(N) «(N, N - 1) 

V(l) = E {x^(N - 1) M(N - L) x(N - 1) + a(N - 1)} (4.4.14) 

where 

M(N - 1) - »^(N, N - 1) P(N) $(N, N - 1) - <t^(N, N - 1) 

• P(N) r(N, N - l)(r’^(N, N - 1) P(N) r^(N, N - 1) 

+ R(N - 1))“^ r^(N, N - 1) P(N) i(N, N - 1) 


(4.4.15) 
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a(N - 1) • E {S(N - N - 1) R(N) r(N. N - 1) • 


(r^(N, H - 1) P(N) r(N, N - 1) + R(N - 1)“^ • 


r^(N, N - 1) P(N) r(N, N - 1)1 R(N - 1) 


+ w(N - 1)^ N - 1)^ P(N) 4»(N. N - 1) w(N - 1)} 

(4. A. 16) 


Double-Stage Problem 

The optioal cost for the double-stage problem Is 

'''21 - mT- 2) u(n"- 1) E 

+ u‘(N - 2) R(S - 2) u(N - 2)1 + [x^(N) P(N) x(N) 

+ u^(N - 2) R(N - 1) u(N - 1)1} (4.4.17) 

From the Principle of Optimality and the expression for V(l) 

^^(2) - , E - 1) Q(N - 1) x(N - 1) 

+ u^(N - 2) R(N - 2) u(N - 2) + V(l)} (4.4.18) 

Fran Equation (4.4.14) 

V(2) - E - 1) x(N - 1) 

u(N - Z) 

+ u^(N - 2) R(N - 2) u(N - 2) + E [x^(N - i) M(N - 1) 


x(N - 1)1 + ot(N - 1) } 


(4.4.19) 
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Hence, Equation (4.4.18) can be simplified as 

V(2) - E {[x^CN - 1)(Q(N - 1) + M(N - 1)) x(N - 1) 

+ u^(N - 2) R(S - 2) u(N - 2)] + ct(N - 1)} (4.4.20) 

Since the choice of u(N - 2) will not affect a(N - 1), only the 
first tvo terms of (4.4.20) will be Involved in optimization. 

Defining P(N - 1) - M(N - 1) + Q(N - 1) , 

V(2) - E {x^(N - 1) P(K - 1) x(N - 1) 

+ u*(N - ^) R(N - 2) u(N - 2) + ot(N - 1); (4.4.21) 

From the equation of our system model, 

x(N - 1) - $(N - 1, N - 2) x(N - 2) + ij;(N - 1, N - 2) w(N - 2) 

+ r(N - 1, N - 2) u(N - 2) (4.4.22) 

Comparing Equations (4.4.21) and (4.4.22) with Equations (4.4.1) and 
(4.4.2), the equations for the double-stage problem are of the same 
form as the equations for the single-stage problem with the excep- 
tion of the additive term a(N - 1) which is unrelated to the 
optimization. The optimization can, therefore, be carried out in 
the same manner as in the single-stage problem. 

The results are 


u(N - 2) = S(N - 2) x(N - 2/y(M - 2), t^_^) 


(4.4.23) 
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S(S - 2) - - (r^(N - 1, M - 2) P(N - 1) r(N - 1, N - 2) + R(N - 2))"* 
• r^(N - 1, N - 2) P(N - 1) i(N - 1. N - 2) (4.4.24) 

V(2) • E {x^(N - 2) M(N « 2) x(N - 2) + a(N - 2)> (4.4.25) 

P(N - 1) - M(J< - 1) + Q(N - 1) (4.4.26) 


where 


M(N - 2) - - 1, N - 2) P(N - 1) ♦(N - 1, N - 2) 


- J^(N - 1, N - 2) P(N - 1) r(N - 1, N - 2) • 


ir^(N - 1, N - 2) P(S - 1) r(N - 1. N - 2) + 


R(N - 2)1"^ r(N - 1, M - 2) P(N - 1) ®(N - 1, N - 2) 

(4.4.27) 


and 


a(N - 2) » E {-x^(N - 2)[$^(N - 1, N - 2) P(N - 1) F(N - 1, N - 2) 
• S(N - 2)1 i(N - 2) + w^(N - 2) i^^(N - 1, N* - 2) 


• P(N - 1) 1((N - 1, N - 2) w(N - 2)} + a(N - 1) 

(4.4.28) 

Utilizing the Principle of Induction, we can prove the Separation 
Theorem for the general N-stage problem by carrying out the deriva- 
tion for the (k-l)-stage and k-stage problem. Ve will first assume 
the recursive relations (4.4.24), (4,4.26), (4.4.27) and (4.4.28) 
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hold for the (k-l)-scage problem and then prove the Separation 
Theorem by shoving that they also hold for k-stage problem. 

(k~l)~Stages 

The optimal strategy at time N - k + 1 for the (k-1) -stage 
process is characterized by the following equations 

u(N • k + 1) - S(N - k + 1) i(N - k + l/y(N - k + 1), 

(4.4.29) 

S(N-fc+l) - - [r^(N-lri-2, N-k+1) P(N-k+2) r(N-k+2, N-k+1) + R(N-k+l)]"^ 
• r^(N-k+2, N-k+1) P(N-fc+2) 4(N-k+2, N-fc+1) 

P(N - k + 2) - M(N - k + 2) + Q(N - k + 2) (4.4.30) 

V(k-l) - E (x^(N-lcfl) M(N-k+l) x(N-fc+l) + a(N-k+l)> (4.4.31) 


where 


M(N-fc+2) - $^(N-k+2, N-k+1) P(N-k+2) ^(N-k+2, N-fc+1) 


• [r^(N-k+2, N-k+1) P(N-k+2) r(N-k+2, N-k+1) + R(N-k+l] 


• r(N-k+2, N-k+1) P(N-k+2) $(N-k+2, N-k+1) (4.4.32) 

and 

a(N-k+l) - E {-x’*' (N-k+1) [$^(N-k+2, N-k+1) P(N-k+2) 


• r(N-k+2, N-k+1) S(N-k+D] x(N-k+l) + (continued) 
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w^(N-k+l) -^^(N-k+2, N-k+l) P(N-k+2, N-k+1) w(N-k+D) 

+ a(N-k+2) (4.4.33) 

k Stages 

Using the Principle of Optimality the cost function for ;-ne 
k-stage problem can be written as 

V(k) • E {x’^(N-j+D Q(N-j+l) x(N-j+l) 

+ u”^(N-j) R(N-j) u(N-j) +V(K-1)1 (4.4.34) 

Combining (4.4.34) and (4.4.31), 

V(k) - E {x''^(N-j+D P(N-j+l) x(N-j+l) 

+ u^(N-j) R(N-j) u(N-j) + a(N-j+l)} (4.4.35) 

where 

P(N-j+l) - M(N-j+l) + Q(N-j+i) 

From the equations of system dynamics 

x(N-i+l) - <t(N-j+l, N-j) x(N-j) 

+ <i»(N-j+l, N-j) w(N-j) 

+ r(N-j+l, N-j) u(N-j) (4.4.36) 

With the exceptions of the difference in time arguments and the 
additive term a(N-j+l) which is unrelated to the minimization, 
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the equations (4.4.33) and (4.4.36) are in the same form as the 
equations (4.4.21) and (4.4.22). The optimization can be carried 
out by repeatir; the similar steps. We will state the results in 
the following theorem. 

Theorem 4.4.1 (The Separation Theorem) 

The optimal strategy for the stochastic linear system with 
random sampling can be implemented by cascading the optimal Kalman 
filter with the optimal feedback gain matrix of the probabilistic 
linear regulator. Tlie parameters for the two parts of the control 
system can be determined separately. The optimal Kalman filter is 
governed by equations (4. 3. 1-4. 3. 3) . The probabilistic linear 
regulator satisfies the following recursive relations: 

P(k+1) - M(k+1) + Q(k+1) (4.4.37) 

S(k) « -[r^Oc+l, k) P(k^l) r(k+l, k) + R(k)]"^ 

r^(k+l, k) P(k+1) <Kk+l, k) (4.4.38) 

M(k) « $^(k+I, k) P(k+1) <J>(k-i-l, k) + 

$^(k+l, k) P(k+1) r(k+l. k) S(k) (4.4.39) 

for k - N - 1, N - 2, . . . ,0. 

If we are only interested in generating the control, the 
term M(k + 1) can be eliminated by substituting M(k +1) = P(k + 1) - 
Q(k + 1) into Equation (4.4.39). This leads us to the corollary: 
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Corollary 4.4.1. The optimal control for the stochastic linear system 
with random sampling can be generated using the following recursive 
relations : 

u(k) - S(k) x(k) (4.4.40) 

S(k) - - [r^(m, k) P(k+1) r(k+l. k) + R(k)]“^ 

r^(k+l, k) P(k+1) ♦(k+1, k) (4.4.41) 

P(k) » »^(k+l, k) P(k+1) $(k+l, k) 

+ ^'^(k+1, k) P(k+1) :(k+l, k) S(k) + Q(k) (4.4.42) 

k-N-1, N-2, . . . ,0. 

With the exception of notatlonal differences. Equations 
(4.4.40-4.4.42) are the same as Equations (2.3.32-2.3.34). This 
result is significant because the optimization problem can be re- 
duced to two separate optimizations whose solutions are in closed 
form. The most important feature of the optimal strategy for the 
stochastic system with random sampling is that the feedback gain 
matrix is independent of the statistics of the additive noise but 
dependent on the statistics of the sampling process, wh reas the 
optimal filter is independent of the weighting matrices in the 
performance measure. The block diagram for the Control system is 
shown in Figure (4.4.1). 
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4 . 5 Numerical Results 

In this section we present some numerical results of optimal 
control and estimation applied to the F8-DFBW aircraft in a noisy 
environment. The aircraft in the noisy environment is modeled as 

X * Ax + Bu + w (4.5.1) 

y “ Hx + 0 (4.5.2) 


where A is (4 x 4) matrix 
B is (4x2) matrix 
w and 0 are (4x1) random vectors 
H is the (4 x 4) identity matrix 
With the exception of additive Gaussian noise terms, the air- 
craft is assumed to be in the same flight configuration as that used 
in chapter 3. We al; o use the same random sampling process. The 
noise processes w and 0 are independent random vectors with zero 
means. The covariance of w is equal to diag. [.04, .04, .04, .04]. 
The covariance of 0 is equal to diag. [.01, .01, .01, .01]. 

The equivalent discrete model for the system (4.5.1) and 
(4.5.2) is given by 


k 


k 


(4.5.3) 
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y(t' ) = tj^) x(tj^) + H ^ 

k 

t'k 

+ H /^ '*’ (^*5.4) 

""k 

But as t’ -► t, the second and third terms approach zero and 

iC K 

<t(t’j^, tj^) approaches unity. We have the following equivalent model 
after dropping the t notation for simplification. 

x(.k+l) = <^(k+l, k) x(k) + r(k+l, k) u(k) + v(k) (4.5.5) 

y(k) = Hx(k) + 0(k) (4.5.6) 


where 


E{v(k) v(k)^} = ^^^k+1’ 

k 


R(t) 6(t-x) = E (w(t) w (t)} 


(4.5.7) 

(4.5.8) 


For the purpose of comparison both the unaugmented system and 
each uniformly-sampled system were simulated. The results are shown 
in Figures (4. 5. 1-4. 5. 4) . A randomly-sampled system is simulated 
with transition probabilities as shown on Figure (3.3.2). Typical 
responses of the randomly-sampled system using optimal gains calcu- 
lated for the uniformly-sampled system are shown in Figures (4.5.5- 
4.5.7). A typical response of the randomly -samp led system with optimal 
gains is shown in Figure (4.4.8). 
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Figure 4.5.6. Response of the Randomly-Sampled System Using 
Optimal Gains Calculated for the Uniformly- 
Sampled System with Sampling Intervals »• 0.03 Sec, 
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Table 4.5.1. Results of Simulation Study on Stochastic System with Random Sampling: 

20 Random Runs of 10-Second Duration. 


Optimal Gain Optimal Gain Optimal Gain 

for 0.02 sec. for 0.03 sec. for 0.10 sec. Unaugmented 

Uniformly- Uniformly- Uniformly- System 

Sampled System Sampled System Sampled System 


Average 

Cost 


0.982 


1.045 


1.029 


1.490 


1.901 





5. EXTENSIONS AND RELATED TOPICS 


5.1 Introduction 

In this chapter the major results will generalized to cover 
a wider range of applications of probabilistic Rlccatl equations. 

The problem can be re-formulated so that the probabilistic Rlccatl 
equations (2.3.47-2.3.48) form the optimal solution of the system 
which switches randomly among NS different configurations. There 
are a variety of causes for the change of configurations In a dynamic 
system. In a large-scale system the failure of an individual sensor 
or actuator may change the system dynamics. The occurrence of a 
failure Is more frequent In this case due to the size and complexity 
of large systems. In a nonlinear system the dynamics can be thought 
to change as the system moves from one linearized region to another. 
In an adaptive-sampling system, each sampling rate results In a 
unique equivalent discrete model determined by Equations (2.2.15- 
2.2.19). The generalized probabilistic Rlccatl equations will be 
presented in Section 5.2. We will discuss failure detection and 
accommodation of large-scale system in Section 5.3. In Section 5.4 
we will show that the solution of the probabilistic Riccati equation 
is the optimal solution for nonlinear systems in which the non- 
linearity is formulated as a semi-Markov switched-linear process. 

The application of stochastic sampling theory to an adaptive-sampling 
system is presented in Section 5.5. 
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5.2 Generalization of the Probabilistic Rlccati Equations 

In this section, we will consider a discrete-time system which 
switches randomly among m possible configurations {($^, Q^) , 

i = 1, . . . , m}. The cost function for the discrete-time system 
is defined as 


N-1 

J - I E {x(N)''^ Sx(N) + I [x(k)’^ Q(k) x(k) + u'^(k) R(k) u(k)]} 

k-0 

(5.2.1) 


where 


Q(k) e {Q^, i = 1, . . . , m^} 

R(k) e {R^, i ■ 1, . . . , m^) 

for k » 1, 2, 3, . . . 


The dynamics of the system can be described as 

x(k+l) » $(k+l, k) x(k) + r(k+l, k) u(k) (5.2.2) 


where 


<5(k+l, k) e i » 1, . . . , m^} 

F(k+1, k) e {r^, i - 1, ...» m^} 

For notational simplicity we will represent the ith configura- 
tion by S£ and the configuration of the system at time k by S(k); i.e. 


Si » (^^, F^, R^, Q^) for i » 1, . 


m 


102 


and 

S(k) = ($(k+l, !.), r(k+l, k), R(k), Q(k)) 
for k * 1, . . . , N . 

If the transition mechanism among m possible configurations can 
be described as a stationary Markov process, i.e. the transition 
mechanism can be represented by the matrix T defined by 

T = {P(S(k) = S^/S(k-1) = S^)} 

then the optimal control is governed by the following recursive 
relations 

J(N-k, N) = x^(N-k) P(k/i) x(N-k) (5.2.3) 

m ^ 

P(k/i) = I T.. {Q. + K^(N-k/j) R. K(N-k/j) 
j=l ^ ^ 

+ [$ + r K(N-k/j)]'^ P(k-l/j) [$ + r K(N-k/.j)]> 

J J J J 

(5.2.4) 

rii n 

K(N-k/i) - - { E T,, [R, + rj P(k-i/j) r ]■ } 
j.l ij J J J 

ni m 

•{ I T., [r; P(k-l/j) $ 1} (5.2.5) 

j»i ^ ^ ^ 

u(N-k/i) = K(N-k/i) x(N-k) (5.2.6) 


The proof is omitted here because it is similar to that in 


Section 2.3. 
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The optimal solution (5 .2. 3-5. 2 .6) was obtained under the 
assumption that the configuration of the system can be identified 
perfectly with at most a one-step delay. If the structure of the 
system cannot be observed perfectly with at most a one-step delay, 
the solution is no longer optimal. This is because with imperfect 
observation of the system, the control must perform dual functions; 
one is to control the state, and the other is to identify the struc- 
ture of the system [31], [32]. The phenomenon that the control 
influences the estimation of the state or the structure of the system 
is called the dual effect. In the randomly-s^pled system there are 
no dual effects since the control strategy does not affect the 
estimation of the state or the determination of the system structure. 
This is also the basic reason for the separation of estimation and 
control in randomly-sampled systems. Unfortunately, these results 
can not be extended optimally to systems in which the structure is 
uncertain. On the other hand, the optimal control for a system 
with dual effects requires unrealistically large amounts of computer 
resources. Therefore, the use of Equations (5. 2. 3-5. 2. 6) as a 
suboptimal strategy should still be considered. 

5.3 Failure Accommodation in Large Scale Systems 

With ever increasing complexity of digital control systems, 
there is a need for design techniques for the control system that will 
respond rapidly and maintain overall system performance when a 
failure occurs. In designing a reliable control system there are 
two considerations: 1. The control system must respond rapidly 
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when a failure occurs; and 2. There must not be a significant 
degradation in performance during normal operation. These two 
considerations are usually conflicting. In the probabilistic 
Riccati Equations (5. 2. 3-5. 2. 6) the trade-off between these con- 
siderations is assessed by the Markov transition probability matrix 
T. Therefore, in a failure-tolerant control system, the Markov 
transition probability must reflect the probability of failure, 
maintenance time and the probability of recovery. From the discussion 
in Section 5.2 we know that the solution given by Equations (5.2.3- 
5.2.6) is optimal only if the system configuration is identified 
with at most a one-step delay. It is, therefore, important to select 
detection filters which can rapidly identify any change in the system 
configuration. The detection filters developed by Beard [33] and 
Jones [34] are very ;<romising. Their work has led to a systematic 
design procedure for the detection of a wide variety of abrupt 
failures in linear time- invariant systems. The failure modes that 
Beard and Jones have considered include actuator and sensor shifts 
and shifts in the matrices A and B. Let us consider using these 
detection filters in a decentralized control system. A decentralized 
control system can be represented as [35] 

m 

x(t) “ Ax(t) + I B u (t) (5.3.1) 

i-1 


t 




I 







y^(t) » CjX(t) 


j » 1, . . . , m 


(5.3.2) 
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where B = (B, , . . . , B ) 

1 m 

C » (C , . . . , C 
1 m 

Suppose we want to detect a failure in the jth actuator, 
which can be modeled as 

m 

x(t) » Ax(t) + I B u. (t) + B v(t) (5.3.3) 

i«l ^ ^ J 

where v(t) is an arbitrary time function. 

The detection filter is governed by the following dynamics in the 
absence of failure and is illustrated in Figure (5.3.1) 

m 

x(t) * Ax(t) + D (y (t) - C x(t)) + I B u (5.3.4) 

3 1 2 i=l ^ ^ 

The objective of the detection filter is to choose the gain 
matrix D. so that the effects of jth actuator failure are accentuated 
in the detection filter residual 


r(t) « x(t) - x(t) (5.3.5) 

From Equations (5.3.3), (5.3.4) and (5.3.5), we know that 
the detection filter residual has the following dynamics in the 
absence of failure 


f » (A - DjCj) r(t) (5.3.6) 

The output residual is given by 

r ' (t) = Cj r(t) 


(5.3.7) 


^ - •< I I JJW^JJWfciiM llllg'Lkyilii HI 4->za^ g.^^ 



Figure 5.3.1. Failure Detection in a Large-Scale 

Control System Using the Beard- Jones 
Failure Detection Filter. 
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The residual dynamics in the presence of the jth actuator failure 
is governed by 

r(t) « v(t) dx 

*"o ^ 

(5.3.8) 

The detection filter residual is contained in the controllable sub- 


space Wj of which is equal to 


.n-1 


W. - [B^, (A-D.C.) B., . . . (A-D.CJ B.] 
3 j ji j jj 3 


(5.3.9) 


The output residual is contained in the subspace spanned by . 
The failure of jth actuator is, therefore, detectable if can be 
chosen so that rank of C.W. is one and the eigenvalues of (A-D.C.) 
are stable. The choice of will Influence the eigenvalues of 
(A-DjC^) and hence the speed of failure detection. This can be 
illustrated if we judiciously chose 


(A-D^C.) » - al 
j 3 


(5.3.10) 


Furthermore, if we assume that v(t) = V, which is a constant, 
then we have 

-at 


r(t) » r(t^) + V ) B^ 


(5.3.11) 


It is apparent now that the greater the detection filter gain 

a, the faster the residual will approach its steady state which is 
V 

equal to — B^ in the direction of B^ . The output residual is equal 

V 

to — CjB, which is in the direction C.B.. Hence, in a noise-free 
j j 3 j 

system, the speed of failure detection can be made as fast as 
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desired by increasing the detection filter gain. This is, however, 
not possible if noise is present because the magnitude of the output 
residual is inversely proportional to the filter gain. 

From the above discussion, we conclude that the optimal 
control strategy determined by Equations (S. 2. 3-5. 2. 6) together with 
the Beard- Jcnes failure detection filter is an interesting and 
logical approach, especially in systems where the noise level is 
relatively low. 

5.4 Applications to Nonlinear Systems 

A general nonlinear system can be represented by 

X “ f(x, u) (5.4.1) 

where x is a vector of system states and u is the vector of inputs. 
The entire state space of interest can be partitioned into a 
number of regions, viz. {S^, i ■ 1, . . . , m}. The dynamics in 
each region can be approximated by a linear model which is derived 
by using a Taylor series expansion in the neighborhood of the 
"center" (x^, u^) of the region. Using a first-order approximation 
as in Section (3.2), we have the following linearized model for 
each region, 

5x ■ A^<5x + B^6u (5.4.2) 


where 
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A. 

1 





(5.4.3) 


T, 3f / N 

B. = -^ (x. , u.) 
i 3u 1 1 


(5.4.4) 


for i = 1, . . . , m 

For each linear model an equivalent linear discrete-time 
model can be calculated of the form 


x(k+l) = $j^(k+l, k) x(k) + r^(k+l, k) u(k) 

for i = 1, . . . , m (5.4-.5) 

As the state of the nonlinear system moves from one region 
to the other, our linear approximation model should change 
accordingly. A nonlinear system can, therefore, be thought of as 
a linear system with time-varying parameters. Several methods have 
been developed to identify these parameters. One well known tech- 
nique is called "Partitioned Adaptive Estimation," first introduced 
by Magill and later refined by Lainiotis [36], [37]. In this 
approach the optimal estimate is a weighted sum of estimates 
derived from a bank of Kalman filters. The weighting coefficients 
are functions of the measurement residuals. The weighting coef- 
ficient corresponding to the actual plant model will approach unity 
while the other coefficients approach zero given sufficient time 
for the identification. The partitioned adaptive control known as 
"Multiple Model Adaptive Control" was developed in the Electronic 
Systems Laboratory of Massachusetts Institute of Technology. 


The method was successful in controlling the nonlinear dynamics of 
the F-8 aircraft in its various flight configurations [38] . These 
methods, however, failed to track nonlinear plants undergoing 
relatively rapid maneuvers as reported in reference [39]. Moose, 
Wang and Zwicke developed a better estimator by assuming that the 
random configuration changes could be modeled as a semi-Markov 
process [AO], [41], [42]. This formulation is exactly the same as 
that presented in Section 5.2. The optimal control, assuming that 
the plant configuration can be identified with at most a one-step 
delay, is given by Equations (5.23-5.2.6). One feature of Equa- 
tions (5. 2. 3-5. 2. 6) is that the optimal control has accounted for 
a possible change of configuration in the future since the feedback 
gain matrix is generated from a recursive probabilistic Riccati 
equation. Unfortunately, a nonlinear plant can seldom be identi- 
fied within one iteration. On the other hand, since dual optimal 
control requires immensely large amounts of computer resources, 
the suboptlmal approach by assuming, valid or not, that the plant 
can be identified with no more than a one-step delay is still 
attractive. 

5.5 Minimization of Information Flow Using Adaptive Sampling 

The exchange of information between microcomputers is an 
item of concern because its cost is high relative to the cost of 
computation within a given micro-computer. This section explains 
an analytical procedure that assesses the trade-off between the 


' I f'..*- ffiniifff . 
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Stability and control objectives of the control system and its 
information exchange requirements. Figure (5.5.1) shows a typical 
trade-off between the objectives of the control system and informa- 
tion exchange requirements. The optimal trade-off involves 
determining the information pattern of the system (i.e. variables 
exchanged between the distributed microcomputer subsystems and the 
frequency of the exchange) . Past research has considered control 
system designs using quadratic optimization for a fixed informa- 
tion pattern P^. For that process, the variable exchanged and the 
frequency of the exchange are fixed and the control system gains 
and filter constants are selected as in classical regulator theory 
to minimize em objective function of the form 

J » /“ (x^(t) Q(t) x(t) + u^(t) R(t) u(t)) dt (5.5.1) 

Since the information pattern P^ specifies the variables 
available for information and the rate of information exchange, the 
result of the minimization of will depend implicitly on P^. 

The variation of the optimal control objective function with P^ is 
recognized and accounted for in the adaptive sampling. This is done 
by considering a total objective function that not only involves 
performance of the objectives of the control system as in classi- 
cal regulator theory but also penalizes the use of information flow. 
The total objective function to be minimized is of the form 

J - J + C(P.) 
o ^ 


Sample time 


Figure 5.5 


1. Qualitative Effect of Information Exchange 
Rate (Sample Rate) on the Stability and 
Control Objective Function J , the 
Information Exchange Penalty Function C, 
and the Total Design Objective Function J. 


The first term forces the algorithm to obtain a desired system 
performance and the second term penalizes high information exchange 
rates . 

For the purpose of optimizing the total objective function, 

J, the sampling rate t is assumed to be in the range of usable 
values. The range of usable x-values is limited from below by 
computer I/O capabilities and is limited from above by stability 
requirements. For the case of a linear plant of the form 

X = Ax(t) + Bu(t) (5.5.2) 

u(t) » u(k) (5.5.3) 

for t, ^ t < t,. , , k = 1, 2, . . . with the total objective function 

iC ““ K*i X 

J » (x^(t) Q x(t) + u^(t) R u(t) dt + f(t) (5.5.4) 

When the plant (5.5.2) is subject to the constraint (5.5.3), 
the plant dynamics have the following equivalent discrete-time 
representation 

x(k+l) - $(k+l, k) x(k) + r(k+l, k) u(k) (5.5.4) 

with the cost function 
00 

J - [x(k) Q(k) x(k) + 2x(k) M(k) x(k) 

k=»l 

+ u(k) R(k) u(k)] + f(t) (5.5.5) 


where <Kk +1, k) , F(k +1, k) , Q(k), M(k) and R(k) are functions 
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of T. The minimization of J should be taken over the set of 
sequences U = {u(k); k “ 1, . . . , This, however, leads to 

a complex analytical problem. The analytical complexity can be 
reduced by considering a restricted class of information patterns 
Pj where the data bus is assumed to be sampled at uniform intervals. 
If the data bus is, in fact, assumed to be sampled at uniform 
intervals, x, the solution to the optimization problem over the 
sequence u is the feedback control law: 

u*(k) » F(t) x(k) (5.5.6) 

where 

F(x) - - (R + r^KT)"^(r^K« + M^) (5.5.7) 

and K is the solution to the matrix Riccatl equation 

K =« + Q - (A<t + M^)^(R + r^KT)”^ (f\$ + M^) (5.5.8) 

Note that F and K, as well as Q, R and M are functions of x. When 
the sequence u (k) Is applied, the cost function J becomes 

J “ x^(o) K(x) x(o) + f(x) (5.5.9) 

which is a function of the initial state x(o) and sample time x. 

If the number of usable sampling intervals is small and the initial 
state is known, the minimal J can be determined by evaluating 
Equation (5.5.9) for all possible values of x. This is not 


115 


feasible for an on-line application if the number of usable sampling 
intervals is large. 

A different approach involves computing t by optimizing 
J over a sample interval (t^^, + b) . The cost accrued over that 

interval is 


AJ(t) ■ X (k) H(t) x(k) + f(x) 


(5.5.10) 


where 


H(t) = Q(t) + 2M(t) F(t) + F^(t) R(t) F(t) 


(5.5.11) 


The functions Q(t), M(t) and R(t) are analytic in t and, 
under the assumption that f(x) is also analytic, AJ(t) can be 
expanded in a Taylor series about the sample time which was last 
used, viz., t^. This results in 

AJ(t) = f + x'^(k)[H + H'(t-t^) + I H"(t-t^)^] x(k) 


+ f'(T-T^) + y f'*(T-T^)^ 


(5.5.12) 


3H 

where e.g., H' A ~ and all functions are evaluated at the last 

oT 

sample time, t^. The minimization can be simplified by neglecting 

2 

higher-order terms (higher than (t-t^) ) . This results in 


T - T 


f'(T^) + X (k) H*(t^) x(k) 

3 * -- ■- »■■■■ 

^ f"(Tp + x^(k) H"(t^) x(k) 


(5.5.12) 


For cn-line implementation, it is only necessary to store f*, f", H' 


and H”. 
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Using the above adaptive-sampling algorithm, the control 
system will have random sampling intervals. Unless the sampling 
process can be modeled as a semi-Markov process, there is currently 
no control law which can account for the random sampling intervals. 
In order to use Equations (5. 2. 3-5. 2. 6) to calculate the control 
law, we must know the transition probabilities. On the other hand, 
in order to determine the transition probabilities in simulation, 
we must first have the control law. To avoid this dilemma, a 
heuristic method is introdu'.ed in the flow chart shown in Figure 
(5.5.2). In this approach the steady-state gain is calculated for 
each sampling interval assuming that the gain will be used in a 
uniformly-sampled system. These gains are then applied to the 
simulation of the adaptively-sampled system. These gains are, of 
course, not optimal in the adaptively-sampled system. From the 
simulation tne Markov transition probabilities between adjacent 
sampling intervals are determined. Using these probabilities, the 
optimal quasi-steady-state gains can be determined from Equations 
(5. 2. 3-5. 2. 6) . We will then iterate between generating the new 
transition probabilities and calculating the quasi-steady gains 
until the algorithm converges. Since this method is heuristic in 
nature, the convergence of the algorithm is not guaranteed. 

However, if the algorithm does converge, the quasi-steady-state 
gains should be very closed to the optimal solution. 


Select a set of usable sampling 
intervals 

S = {Tj^, . . . , T^} 


Calculate the uniformly-sampled steady- 
state gain for each Tj , 

{G!, .... Gl> 


Use gains {G^, G^^} to 

simulate the adaptive sampling 
system to determine the tran- 
sition probabilities 

P . } = P^(t = T./ » T.) 

XT n IT, 1 


Solve for the quasi-steady- 
state gains using the stochas- 
tic Riccati equations based oi 
the above probabilities, i.e. 

{Gfl 

1 m 


Li-i. 


Figure 5.5.2, A Heuristic Approach to Determine the 

Control Law for an Adaptively-Sampled System 
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